2024-09-10T03:46:03.447 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T03:46:03.461 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T03:46:03.538 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607 branch: wip-adk-testing-2024-09-09-1904 description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} email: adking@redhat.com first_in_suite: false job_id: '7898607' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-adk-testing-2024-09-09-1904 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - CEPHADM_AGENT_DOWN log-only-match: - CEPHADM_ sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 workunit: branch: wip-adk-testing-2024-09-09-1904 sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c owner: scheduled_adking@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 6964 sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c sleep_before_teardown: 0 subset: 1/20 suite: orch:cephadm suite_branch: wip-adk-testing-2024-09-09-1904 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5puBFFRaOGubEKHNzfv/EEUVJp09PGY3NsNpMUswWOjvWvmHK0VR5hC1/lTvh2KJdAwjyb0KeRAsBT+qwWVB0= smithi050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKM/RYd6NuZmwfUO3ZPSFWAvhlw/A0+yCxW7k6V3H50tKE4fRfRtqmeVSTlGky5ytH8PirAgGRtsEnn0h3bJS5Q= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_02:59:23 tube: smithi user: adking verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T03:46:03.538 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa; will attempt to use it 2024-09-10T03:46:03.539 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_3a5e3f16720f432aea965c40a8f6eb480095ec6c/qa/tasks 2024-09-10T03:46:03.539 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T03:46:03.541 INFO:teuthology.task.internal:Checking packages... 2024-09-10T03:46:03.563 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3a5e3f16720f432aea965c40a8f6eb480095ec6c' 2024-09-10T03:46:03.563 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T03:46:03.563 INFO:teuthology.packaging:ref: None 2024-09-10T03:46:03.563 INFO:teuthology.packaging:tag: None 2024-09-10T03:46:03.563 INFO:teuthology.packaging:branch: wip-adk-testing-2024-09-09-1904 2024-09-10T03:46:03.563 INFO:teuthology.packaging:sha1: 3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:46:03.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-adk-testing-2024-09-09-1904 2024-09-10T03:46:03.701 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4885.g3a5e3f16 2024-09-10T03:46:03.703 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T03:46:03.709 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T03:46:03.709 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T03:46:03.715 INFO:teuthology.task.internal:Saving configuration 2024-09-10T03:46:03.730 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T03:46:03.737 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T03:46:03.758 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 03:41:14.276550', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG5puBFFRaOGubEKHNzfv/EEUVJp09PGY3NsNpMUswWOjvWvmHK0VR5hC1/lTvh2KJdAwjyb0KeRAsBT+qwWVB0='} 2024-09-10T03:46:03.779 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi050.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 03:41:14.278229', 'locked_by': 'scheduled_adking@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKM/RYd6NuZmwfUO3ZPSFWAvhlw/A0+yCxW7k6V3H50tKE4fRfRtqmeVSTlGky5ytH8PirAgGRtsEnn0h3bJS5Q='} 2024-09-10T03:46:03.780 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T03:46:03.785 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T03:46:03.786 INFO:teuthology.task.internal:roles: ubuntu@smithi050.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T03:46:03.786 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T03:46:03.837 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5b48cebbe0>, signals=[15]) 2024-09-10T03:46:03.837 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T03:46:03.843 INFO:teuthology.task.internal:Opening connections... 2024-09-10T03:46:03.843 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-09-10T03:46:03.845 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:46:03.921 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi050.front.sepia.ceph.com 2024-09-10T03:46:03.922 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:46:03.998 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T03:46:04.004 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-09-10T03:46:04.022 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-09-10T03:46:04.022 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-09-10T03:46:04.081 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T03:46:04.082 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T03:46:04.082 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T03:46:04.082 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-09-10T03:46:04.100 DEBUG:teuthology.orchestra.run.smithi050:> uname -m 2024-09-10T03:46:04.118 INFO:teuthology.orchestra.run.smithi050.stdout:x86_64 2024-09-10T03:46:04.118 DEBUG:teuthology.orchestra.run.smithi050:> cat /etc/os-release 2024-09-10T03:46:04.176 INFO:teuthology.orchestra.run.smithi050.stdout:NAME="CentOS Stream" 2024-09-10T03:46:04.176 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION="9" 2024-09-10T03:46:04.176 INFO:teuthology.orchestra.run.smithi050.stdout:ID="centos" 2024-09-10T03:46:04.176 INFO:teuthology.orchestra.run.smithi050.stdout:ID_LIKE="rhel fedora" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:VERSION_ID="9" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:PLATFORM_ID="platform:el9" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:ANSI_COLOR="0;31" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:LOGO="fedora-logo-icon" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:HOME_URL="https://centos.org/" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T03:46:04.177 INFO:teuthology.orchestra.run.smithi050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T03:46:04.178 INFO:teuthology.lock.ops:Updating smithi050.front.sepia.ceph.com on lock server 2024-09-10T03:46:04.200 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T03:46:04.208 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T03:46:04.213 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T03:46:04.214 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-09-10T03:46:04.216 DEBUG:teuthology.orchestra.run.smithi050:> test '!' -e /home/ubuntu/cephtest 2024-09-10T03:46:04.233 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T03:46:04.238 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T03:46:04.238 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-09-10T03:46:04.274 DEBUG:teuthology.orchestra.run.smithi050:> test -z $(ls -A /var/lib/ceph) 2024-09-10T03:46:04.301 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T03:46:04.359 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T03:46:04.373 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T03:46:04.373 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T03:46:04.373 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T03:46:04.374 DEBUG:teuthology.orchestra.run.smithi050:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T03:46:04.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:46:04.391 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-10T03:46:04.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:46:04.392 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-10T03:46:04.447 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-503.el9.x86_64 2024-09-10T03:46:04.448 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-503.el9.x86_64 2024-09-10T03:46:04.448 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-09-10T03:46:04.450 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-503.el9.x86_64 2024-09-10T03:46:04.450 INFO:teuthology.task.kernel:Running kernel on smithi050: 5.14.0-503.el9.x86_64 2024-09-10T03:46:04.450 DEBUG:teuthology.orchestra.run.smithi050:> sudo yum install -y kernel 2024-09-10T03:46:07.051 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - BaseOS 8.9 MB/s | 8.2 MB 00:00 2024-09-10T03:46:08.098 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.2 MB 00:01 2024-09-10T03:46:09.378 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - AppStream 17 MB/s | 20 MB 00:01 2024-09-10T03:46:10.117 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 24 MB/s | 20 MB 00:00 2024-09-10T03:46:15.475 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - CRB 3.0 MB/s | 6.5 MB 00:02 2024-09-10T03:46:17.205 INFO:teuthology.orchestra.run.smithi050.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 19 kB 00:00 2024-09-10T03:46:18.342 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 6.5 MB 00:04 2024-09-10T03:46:19.801 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 92 kB/s | 19 kB 00:00 2024-09-10T03:46:20.019 INFO:teuthology.orchestra.run.smithi050.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 23 MB 00:02 2024-09-10T03:46:21.031 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-10T03:46:25.523 INFO:teuthology.orchestra.run.smithi050.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T03:46:26.653 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T03:46:27.646 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T03:46:27.646 INFO:teuthology.orchestra.run.smithi050.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T03:46:27.739 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:46:27.748 INFO:teuthology.orchestra.run.smithi050.stdout:Install 4 Packages 2024-09-10T03:46:27.749 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:46:27.749 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 79 M 2024-09-10T03:46:27.749 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 126 M 2024-09-10T03:46:27.749 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-09-10T03:46:27.906 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-10T03:46:27.908 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-10T03:46:27.953 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-10T03:46:27.989 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:46:27.999 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:46:28.006 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:46:28.049 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.058 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.066 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.151 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.159 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.167 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.194 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.201 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.210 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.269 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.275 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.281 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.366 INFO:teuthology.orchestra.run.smithi050.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 44 kB/s | 23 kB 00:00 2024-09-10T03:46:28.470 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.20) 2024-09-10T03:46:28.523 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://us.mirrors.cicku.me/centos/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 172.65.90.22) 2024-09-10T03:46:28.553 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.613 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirror.team-cymru.com/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 216.31.2.234) 2024-09-10T03:46:28.633 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for https://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.654 INFO:teuthology.orchestra.run.smithi050.stdout:[MIRROR] kernel-modules-core-5.14.0-505.el9.x86_64.rpm: Status code: 404 for http://mirrors.iu13.net/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-core-5.14.0-505.el9.x86_64.rpm (IP: 206.82.17.213) 2024-09-10T03:46:28.777 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T03:46:28.777 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T03:46:28.872 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-10T03:46:28.888 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:46:28.888 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-10T03:46:28.888 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:46:28.888 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-10T03:46:28.888 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2024-09-10T03:46:28.889 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:46:28.890 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 79 M 2024-09-10T03:46:28.890 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 126 M 2024-09-10T03:46:28.890 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-10T03:46:29.072 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 292 kB/s | 23 kB 00:00 2024-09-10T03:46:29.481 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-09-10T03:46:29.864 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 40 MB/s | 35 MB 00:00 2024-09-10T03:46:30.473 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 20 MB/s | 29 MB 00:01 2024-09-10T03:46:30.473 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:46:30.473 INFO:teuthology.orchestra.run.smithi038.stdout:Total 50 MB/s | 79 MB 00:01 2024-09-10T03:46:31.183 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-10T03:46:31.492 INFO:teuthology.orchestra.run.smithi050.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 4.3 MB/s | 16 MB 00:03 2024-09-10T03:46:31.590 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-10T03:46:31.591 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-10T03:46:32.584 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-10T03:46:32.584 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-10T03:46:34.591 INFO:teuthology.orchestra.run.smithi050.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.6 MB/s | 29 MB 00:06 2024-09-10T03:46:35.155 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-10T03:46:35.836 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:46:35.943 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:46:36.683 INFO:teuthology.orchestra.run.smithi050.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 3.9 MB/s | 35 MB 00:08 2024-09-10T03:46:36.688 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:46:36.689 INFO:teuthology.orchestra.run.smithi050.stdout:Total 8.8 MB/s | 79 MB 00:08 2024-09-10T03:46:37.394 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-09-10T03:46:37.779 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-09-10T03:46:37.780 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-09-10T03:46:38.754 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:46:38.776 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-09-10T03:46:38.777 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-09-10T03:46:38.879 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:46:41.370 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-09-10T03:46:42.038 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:46:42.128 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:46:44.000 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:46:44.038 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:46:44.936 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:46:45.073 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:46:49.194 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:46:50.195 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:46:50.241 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:46:55.381 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:27.954 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:27.995 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:28.390 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:28.390 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:47:28.390 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:47:28.391 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.207 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.208 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:47:29.208 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-10T03:47:29.573 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:47:29.656 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.657 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:47:29.657 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T03:47:29.657 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T03:47:29.657 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T03:47:29.657 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T03:47:29.657 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T03:47:29.657 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:47:29.715 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:29.716 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:47:29.716 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 82.el9 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8274670 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T03:47:29.756 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:47:29.757 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T03:47:29.758 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T03:47:29.758 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 5.14.0-505.el9.x86_64 2024-09-10T03:47:29.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T03:47:30.374 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-09-10T03:47:31.791 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T03:47:31.820 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-09-10T03:47:31.827 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-09-10T03:47:31.842 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.m6ACUlRnIC 2024-09-10T03:47:31.843 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.m6ACUlRnIC 2024-09-10T03:47:31.908 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.m6ACUlRnIC 2024-09-10T03:47:32.051 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.m6ACUlRnIC is 6KB 2024-09-10T03:47:32.099 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.m6ACUlRnIC 2024-09-10T03:47:32.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T03:47:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T03:47:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T03:47:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T03:47:32.176 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T03:47:32.178 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T03:47:32.333 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-09-10T03:47:34.202 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:34.244 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:34.550 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:34.551 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T03:47:34.551 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T03:47:34.551 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:47:35.395 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-09-10T03:47:35.728 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:47:35.815 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.816 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:47:35.816 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T03:47:35.816 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T03:47:35.816 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T03:47:35.816 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T03:47:35.816 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi050.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T03:47:35.816 DEBUG:teuthology.orchestra.run.smithi050:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T03:47:35.875 INFO:teuthology.orchestra.run.smithi050.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T03:47:35.875 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T03:47:35.875 DEBUG:teuthology.orchestra.run.smithi050:> sudo rpm -qi grub2-tools 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Name : grub2-tools 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Epoch : 1 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Version : 2.06 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Release : 82.el9 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Architecture: x86_64 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Group : Unspecified 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Size : 8274670 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:License : GPLv3+ 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T03:47:35.906 INFO:teuthology.orchestra.run.smithi050.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Packager : builder@centos.org 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Vendor : CentOS 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Summary : Support tools for GRUB. 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:Description : 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout:hardware devices. 2024-09-10T03:47:35.907 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:47:35.908 INFO:teuthology.orchestra.run.smithi050.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T03:47:35.908 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T03:47:35.908 INFO:teuthology.task.kernel:Updating grub on smithi050 to boot 5.14.0-505.el9.x86_64 2024-09-10T03:47:35.909 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T03:47:36.488 INFO:teuthology.orchestra.run.smithi050.stderr:Generating grub configuration file ... 2024-09-10T03:47:37.799 INFO:teuthology.orchestra.run.smithi050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T03:47:37.829 INFO:teuthology.orchestra.run.smithi050.stderr:done 2024-09-10T03:47:37.831 DEBUG:teuthology.orchestra.run.smithi050:> mktemp 2024-09-10T03:47:37.847 INFO:teuthology.orchestra.run.smithi050.stdout:/tmp/tmp.nmgI5KTcZk 2024-09-10T03:47:37.847 DEBUG:teuthology.orchestra.run.smithi050:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.nmgI5KTcZk 2024-09-10T03:47:37.913 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0666 /tmp/tmp.nmgI5KTcZk 2024-09-10T03:47:38.063 DEBUG:teuthology.orchestra.remote:smithi050:/tmp/tmp.nmgI5KTcZk is 6KB 2024-09-10T03:47:38.110 DEBUG:teuthology.orchestra.run.smithi050:> rm -fr /tmp/tmp.nmgI5KTcZk 2024-09-10T03:47:38.124 DEBUG:teuthology.orchestra.run.smithi050:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T03:47:38.187 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T03:47:38.187 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T03:47:38.187 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T03:47:38.187 INFO:teuthology.orchestra.run.smithi050.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T03:47:38.188 DEBUG:teuthology.orchestra.run.smithi050:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T03:47:38.336 DEBUG:teuthology.orchestra.run.smithi050:> sudo shutdown -r now 2024-09-10T03:48:02.358 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T03:48:02.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-10T03:48:02.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:48:08.341 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T03:48:08.342 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-10T03:48:08.342 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:49:02.413 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T03:49:08.349 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T03:49:11.417 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-10T03:49:11.418 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:49:15.096 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-10T03:49:15.514 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-10T03:49:15.514 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T03:49:15.514 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-10T03:49:15.529 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-505.el9.x86_64 2024-09-10T03:49:15.530 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T03:49:15.530 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T03:49:15.530 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T03:49:16.530 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T03:49:16.531 DEBUG:teuthology.orchestra.run.smithi038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T03:49:16.601 INFO:teuthology.orchestra.run.smithi038.stdout:ttyS1 2024-09-10T03:49:16.620 DEBUG:teuthology.parallel:result is None 2024-09-10T03:49:17.350 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-10T03:49:17.351 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:49:17.760 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-10T03:49:18.171 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-10T03:49:18.171 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T03:49:18.172 DEBUG:teuthology.orchestra.run.smithi050:> uname -r 2024-09-10T03:49:18.187 INFO:teuthology.orchestra.run.smithi050.stdout:5.14.0-505.el9.x86_64 2024-09-10T03:49:18.187 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T03:49:18.187 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T03:49:18.187 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T03:49:19.188 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T03:49:19.188 DEBUG:teuthology.orchestra.run.smithi050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T03:49:19.264 INFO:teuthology.orchestra.run.smithi050.stdout:ttyS1 2024-09-10T03:49:19.298 DEBUG:teuthology.parallel:result is None 2024-09-10T03:49:19.298 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T03:49:19.430 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T03:49:19.431 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T03:49:19.434 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T03:49:19.450 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T03:49:19.457 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T03:49:19.507 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T03:49:19.507 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T03:49:19.509 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T03:49:19.537 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T03:49:19.544 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T03:49:19.545 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T03:49:19.569 DEBUG:teuthology.orchestra.run.smithi050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T03:49:19.594 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:49:19.600 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:49:19.605 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:49:19.612 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T03:49:19.613 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T03:49:19.623 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T03:49:19.624 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T03:49:19.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T03:49:19.680 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T03:49:19.690 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T03:49:19.691 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T03:49:19.724 DEBUG:teuthology.orchestra.run.smithi050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T03:49:19.739 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:49:19.829 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:49:19.874 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:49:19.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:49:20.006 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:49:20.006 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T03:49:20.075 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:49:20.115 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T03:49:20.164 DEBUG:teuthology.orchestra.run.smithi050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:49:20.246 DEBUG:teuthology.orchestra.run.smithi050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T03:49:20.288 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:49:20.288 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T03:49:20.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-09-10T03:49:20.355 DEBUG:teuthology.orchestra.run.smithi050:> sudo service rsyslog restart 2024-09-10T03:49:20.411 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T03:49:20.442 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T03:49:20.809 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T03:49:20.817 INFO:teuthology.task.internal:Starting timer... 2024-09-10T03:49:20.817 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T03:49:20.832 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T03:49:20.847 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T03:49:20.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-09-10T03:49:20.919 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-09-10T03:49:20.921 DEBUG:teuthology.orchestra.run.smithi050:> sudo service auditd rotate 2024-09-10T03:49:20.985 INFO:teuthology.orchestra.run.smithi050.stdout:Rotating logs: 2024-09-10T03:49:20.987 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T03:49:20.987 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-09-10T03:49:21.024 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-09-10T03:49:21.024 DEBUG:teuthology.orchestra.run.smithi050:> /usr/sbin/getenforce 2024-09-10T03:49:21.049 INFO:teuthology.orchestra.run.smithi050.stdout:Permissive 2024-09-10T03:49:21.049 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi050.front.sepia.ceph.com': 'permissive'} 2024-09-10T03:49:21.049 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T03:49:21.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:49:21.094 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T03:49:21.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T03:49:21.121 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T03:49:21.121 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-09-10T03:49:21.214 DEBUG:teuthology.orchestra.run.smithi050:> sudo /usr/sbin/setenforce permissive 2024-09-10T03:49:21.263 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T03:49:21.276 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T03:49:21.279 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T03:49:21.417 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T03:49:21.427 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T03:49:21.428 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T03:53:14.220 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi050.front.sepia.ceph.com')] 2024-09-10T03:53:14.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-10T03:53:14.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:53:14.303 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-10T03:53:14.385 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-10T03:53:14.385 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-10T03:53:14.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T03:53:14.464 DEBUG:teuthology.orchestra.run.smithi050:> true 2024-09-10T03:53:14.542 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi050.front.sepia.ceph.com' 2024-09-10T03:53:14.542 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T03:53:14.553 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T03:53:14.553 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T03:53:14.554 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T03:53:14.556 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T03:53:14.556 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T03:53:14.586 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T03:53:14.602 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T03:53:14.615 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T03:53:14.627 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-09-10T03:53:14.631 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T03:53:14.639 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-09-10T03:53:14.654 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T03:53:14.659 INFO:teuthology.orchestra.run.smithi050.stderr:sudo: ntpd: command not found 2024-09-10T03:53:14.669 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T03:53:14.671 INFO:teuthology.orchestra.run.smithi050.stdout:506 Cannot talk to daemon 2024-09-10T03:53:14.686 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T03:53:14.701 INFO:teuthology.orchestra.run.smithi050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T03:53:14.736 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.740 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.756 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.760 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T03:53:14.761 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T03:53:14.771 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T03:53:14.771 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi038.front.sepia.ceph.com 2024-09-10T03:53:14.771 DEBUG:teuthology.orchestra.run.smithi038:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T03:53:14.772 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi050.front.sepia.ceph.com 2024-09-10T03:53:14.772 DEBUG:teuthology.orchestra.run.smithi050:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T03:53:15.145 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:Removing: 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout:Removing unused dependencies: 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T03:53:15.159 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout:Remove 2 Packages 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout:Freed space: 5.5 M 2024-09-10T03:53:15.160 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-10T03:53:15.164 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-10T03:53:15.164 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-10T03:53:15.171 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout:Removing: 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout:Removing unused dependencies: 2024-09-10T03:53:15.185 INFO:teuthology.orchestra.run.smithi050.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout:Remove 2 Packages 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout:Freed space: 5.5 M 2024-09-10T03:53:15.186 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-09-10T03:53:15.190 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-09-10T03:53:15.190 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-09-10T03:53:15.241 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-10T03:53:15.241 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-10T03:53:15.255 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-09-10T03:53:15.256 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-09-10T03:53:15.397 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-10T03:53:15.402 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-09-10T03:53:15.480 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T03:53:15.482 INFO:teuthology.orchestra.run.smithi050.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T03:53:15.534 INFO:teuthology.orchestra.run.smithi038.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T03:53:15.537 INFO:teuthology.orchestra.run.smithi050.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T03:53:15.926 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T03:53:15.926 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T03:53:16.116 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T03:53:16.117 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T03:53:16.203 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T03:53:16.204 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:16.204 INFO:teuthology.orchestra.run.smithi050.stdout:Removed: 2024-09-10T03:53:16.204 INFO:teuthology.orchestra.run.smithi050.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T03:53:16.204 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:16.204 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-09-10T03:53:16.391 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T03:53:16.391 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:16.391 INFO:teuthology.orchestra.run.smithi038.stdout:Removed: 2024-09-10T03:53:16.391 INFO:teuthology.orchestra.run.smithi038.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T03:53:16.392 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:16.392 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-10T03:53:16.999 INFO:teuthology.orchestra.run.smithi050.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 03:52:25 AM UTC. 2024-09-10T03:53:17.137 INFO:teuthology.orchestra.run.smithi050.stdout:Dependencies resolved. 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi050.stdout: Package Architecture Version Repository Size 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:17.138 INFO:teuthology.orchestra.run.smithi050.stdout:Installing: 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout:Installing dependencies: 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction Summary 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout:================================================================================ 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout:Install 6 Packages 2024-09-10T03:53:17.139 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:17.148 INFO:teuthology.orchestra.run.smithi050.stdout:Total download size: 2.0 M 2024-09-10T03:53:17.148 INFO:teuthology.orchestra.run.smithi050.stdout:Installed size: 9.2 M 2024-09-10T03:53:17.148 INFO:teuthology.orchestra.run.smithi050.stdout:Downloading Packages: 2024-09-10T03:53:17.273 INFO:teuthology.orchestra.run.smithi038.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 03:52:25 AM UTC. 2024-09-10T03:53:17.409 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-10T03:53:17.409 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:17.409 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-10T03:53:17.409 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout:Install 6 Packages 2024-09-10T03:53:17.410 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:17.414 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 2.0 M 2024-09-10T03:53:17.414 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 9.2 M 2024-09-10T03:53:17.414 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-10T03:53:17.787 INFO:teuthology.orchestra.run.smithi038.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 384 kB/s | 44 kB 00:00 2024-09-10T03:53:17.821 INFO:teuthology.orchestra.run.smithi038.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 709 kB/s | 106 kB 00:00 2024-09-10T03:53:17.832 INFO:teuthology.orchestra.run.smithi050.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 76 kB/s | 44 kB 00:00 2024-09-10T03:53:17.855 INFO:teuthology.orchestra.run.smithi038.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.8 MB/s | 906 kB 00:00 2024-09-10T03:53:17.880 INFO:teuthology.orchestra.run.smithi038.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 777 kB/s | 72 kB 00:00 2024-09-10T03:53:17.905 INFO:teuthology.orchestra.run.smithi038.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.0 MB/s | 84 kB 00:00 2024-09-10T03:53:17.941 INFO:teuthology.orchestra.run.smithi050.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 154 kB/s | 106 kB 00:00 2024-09-10T03:53:17.955 INFO:teuthology.orchestra.run.smithi038.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.2 MB/s | 837 kB 00:00 2024-09-10T03:53:17.956 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:17.956 INFO:teuthology.orchestra.run.smithi038.stdout:Total 3.7 MB/s | 2.0 MB 00:00 2024-09-10T03:53:18.033 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-10T03:53:18.048 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-10T03:53:18.048 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-10T03:53:18.233 INFO:teuthology.orchestra.run.smithi050.stdout:(3/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 922 kB/s | 906 kB 00:00 2024-09-10T03:53:18.350 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-10T03:53:18.350 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-10T03:53:18.525 INFO:teuthology.orchestra.run.smithi050.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 104 kB/s | 72 kB 00:00 2024-09-10T03:53:18.625 INFO:teuthology.orchestra.run.smithi050.stdout:(5/6): python3-kmod-0.9-32.el9.x86_64.rpm 123 kB/s | 84 kB 00:00 2024-09-10T03:53:18.779 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-10T03:53:18.868 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T03:53:18.907 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T03:53:18.947 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T03:53:19.176 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T03:53:19.211 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T03:53:19.699 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T03:53:19.730 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T03:53:19.784 INFO:teuthology.orchestra.run.smithi050.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 540 kB/s | 837 kB 00:01 2024-09-10T03:53:19.784 INFO:teuthology.orchestra.run.smithi050.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:53:19.784 INFO:teuthology.orchestra.run.smithi050.stdout:Total 777 kB/s | 2.0 MB 00:02 2024-09-10T03:53:19.869 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction check 2024-09-10T03:53:19.885 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction check succeeded. 2024-09-10T03:53:19.885 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction test 2024-09-10T03:53:20.182 INFO:teuthology.orchestra.run.smithi050.stdout:Transaction test succeeded. 2024-09-10T03:53:20.183 INFO:teuthology.orchestra.run.smithi050.stdout:Running transaction 2024-09-10T03:53:20.637 INFO:teuthology.orchestra.run.smithi050.stdout: Preparing : 1/1 2024-09-10T03:53:20.722 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T03:53:20.722 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T03:53:20.723 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T03:53:20.723 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T03:53:20.723 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T03:53:20.723 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T03:53:20.763 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T03:53:20.809 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T03:53:20.842 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T03:53:21.065 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T03:53:21.080 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T03:53:21.080 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:21.080 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-10T03:53:21.080 INFO:teuthology.orchestra.run.smithi038.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T03:53:21.080 INFO:teuthology.orchestra.run.smithi038.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T03:53:21.081 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T03:53:21.081 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:53:21.081 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-10T03:53:21.097 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T03:53:21.437 DEBUG:teuthology.parallel:result is None 2024-09-10T03:53:21.582 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T03:53:21.608 INFO:teuthology.orchestra.run.smithi050.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T03:53:22.632 INFO:teuthology.orchestra.run.smithi050.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T03:53:22.632 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T03:53:22.632 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T03:53:22.632 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T03:53:22.632 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T03:53:22.633 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout:Installed: 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T03:53:22.981 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:53:22.982 INFO:teuthology.orchestra.run.smithi050.stdout:Complete! 2024-09-10T03:53:23.264 DEBUG:teuthology.parallel:result is None 2024-09-10T03:53:23.264 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-10T03:53:23.277 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-10T03:53:23.278 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:53:23.278 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:53:23.295 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T03:53:23.295 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:23.351 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:53:20.078944308 +0000 2024-09-10T03:53:23.352 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.352 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.352 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.352 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T03:53:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:53:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:53:23.418 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000296362 s, 1.7 MB/s 2024-09-10T03:53:23.419 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T03:53:23.476 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 974 Links: 1 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:53:20.079944320 +0000 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.533 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.534 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T03:53:23.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:53:23.597 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:53:23.597 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000268444 s, 1.9 MB/s 2024-09-10T03:53:23.598 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T03:53:23.655 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-09-10T03:53:23.711 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T03:53:23.711 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 971 Links: 1 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:53:20.079944320 +0000 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.712 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:19.820941189 +0000 2024-09-10T03:53:23.712 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T03:53:23.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:53:23.776 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:53:23.776 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.00026839 s, 1.9 MB/s 2024-09-10T03:53:23.777 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T03:53:23.834 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-09-10T03:53:23.890 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:53:20.079944320 +0000 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.891 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:19.821941202 +0000 2024-09-10T03:53:23.891 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T03:53:23.954 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:53:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:53:23.955 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000268238 s, 1.9 MB/s 2024-09-10T03:53:23.956 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T03:53:24.012 DEBUG:teuthology.orchestra.run.smithi038:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-10T03:53:24.160 INFO:teuthology.orchestra.run.smithi038.stdout:loop 2024-09-10T03:53:24.162 INFO:tasks.nvme_loop:Connecting nvme_loop smithi038:/dev/vg_nvme/lv_1... 2024-09-10T03:53:24.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-10T03:53:24.196 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-10T03:53:24.227 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/vg_nvme/lv_11 2024-09-10T03:53:24.273 INFO:teuthology.orchestra.run.smithi038.stdout:connecting to device: nvme1 2024-09-10T03:53:24.275 INFO:tasks.nvme_loop:Connecting nvme_loop smithi038:/dev/vg_nvme/lv_2... 2024-09-10T03:53:24.275 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-10T03:53:24.354 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-10T03:53:24.386 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/vg_nvme/lv_21 2024-09-10T03:53:24.417 INFO:teuthology.orchestra.run.smithi038.stdout:connecting to device: nvme2 2024-09-10T03:53:24.419 INFO:tasks.nvme_loop:Connecting nvme_loop smithi038:/dev/vg_nvme/lv_3... 2024-09-10T03:53:24.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-10T03:53:24.494 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-10T03:53:24.525 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/vg_nvme/lv_31 2024-09-10T03:53:24.560 INFO:teuthology.orchestra.run.smithi038.stdout:connecting to device: nvme3 2024-09-10T03:53:24.562 INFO:tasks.nvme_loop:Connecting nvme_loop smithi038:/dev/vg_nvme/lv_4... 2024-09-10T03:53:24.562 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-10T03:53:24.636 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-10T03:53:24.667 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/vg_nvme/lv_41 2024-09-10T03:53:24.699 INFO:teuthology.orchestra.run.smithi038.stdout:connecting to device: nvme4 2024-09-10T03:53:24.701 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:53:24.701 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:53:24.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo nvme list -o json 2024-09-10T03:53:24.825 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-10T03:53:24.825 INFO:teuthology.orchestra.run.smithi038.stdout: "Devices":[ 2024-09-10T03:53:24.827 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-10T03:53:24.827 INFO:teuthology.orchestra.run.smithi038.stdout: "NameSpace":1, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "GenericPath":"/dev/ng4n1", 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "SerialNumber":"f3a6674ebb3c7a1c1136", 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "SectorSize":512 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "NameSpace":1, 2024-09-10T03:53:24.828 INFO:teuthology.orchestra.run.smithi038.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "GenericPath":"/dev/ng3n1", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "SerialNumber":"15503f945018a7b16b2d", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "SectorSize":512 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "NameSpace":1, 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "GenericPath":"/dev/ng2n1", 2024-09-10T03:53:24.829 INFO:teuthology.orchestra.run.smithi038.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "SerialNumber":"6ae6bc22a0fc64d8701b", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "SectorSize":512 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "NameSpace":1, 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "GenericPath":"/dev/ng1n1", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "SerialNumber":"ae77412bb688adfead48", 2024-09-10T03:53:24.830 INFO:teuthology.orchestra.run.smithi038.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "SectorSize":512 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "NameSpace":1, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "GenericPath":"/dev/ng0n1", 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "Firmware":"QDV1013D", 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "ModelNumber":"INTEL SSDPEDKE040T7", 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "SerialNumber":"BTLE736103BB4P0KGN", 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "UsedBytes":4000787030016, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "MaximumLBA":7814037168, 2024-09-10T03:53:24.831 INFO:teuthology.orchestra.run.smithi038.stdout: "PhysicalSize":4000787030016, 2024-09-10T03:53:24.832 INFO:teuthology.orchestra.run.smithi038.stdout: "SectorSize":512 2024-09-10T03:53:24.832 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-10T03:53:24.832 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-10T03:53:24.832 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-10T03:53:24.833 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T03:53:24.833 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:53:24.833 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/scratch_devs 2024-09-10T03:53:24.898 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:53:24.899 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:53:24.919 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T03:53:24.919 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_1 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:53:22.003607094 +0000 2024-09-10T03:53:24.977 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:24.978 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:24.978 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:24.978 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T03:53:25.046 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:53:25.046 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:53:25.046 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000109275 s, 4.7 MB/s 2024-09-10T03:53:25.047 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T03:53:25.106 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_2 2024-09-10T03:53:25.162 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 963 Links: 1 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:53:22.004607070 +0000 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.163 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.163 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T03:53:25.228 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:53:25.228 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:53:25.229 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 8.7112e-05 s, 5.9 MB/s 2024-09-10T03:53:25.230 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T03:53:25.289 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_3 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:53:22.004607070 +0000 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.346 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.347 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.347 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T03:53:25.412 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:53:25.412 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:53:25.412 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000107327 s, 4.8 MB/s 2024-09-10T03:53:25.413 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T03:53:25.470 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/vg_nvme/lv_4 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:53:22.004607070 +0000 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.527 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:21.700614367 +0000 2024-09-10T03:53:25.528 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T03:53:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:53:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:53:25.592 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000105108 s, 4.9 MB/s 2024-09-10T03:53:25.594 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T03:53:25.650 DEBUG:teuthology.orchestra.run.smithi050:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-10T03:53:25.809 INFO:teuthology.orchestra.run.smithi050.stdout:loop 2024-09-10T03:53:25.810 INFO:tasks.nvme_loop:Connecting nvme_loop smithi050:/dev/vg_nvme/lv_1... 2024-09-10T03:53:25.811 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-10T03:53:25.845 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-10T03:53:25.875 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/vg_nvme/lv_11 2024-09-10T03:53:25.912 INFO:teuthology.orchestra.run.smithi050.stdout:connecting to device: nvme1 2024-09-10T03:53:25.915 INFO:tasks.nvme_loop:Connecting nvme_loop smithi050:/dev/vg_nvme/lv_2... 2024-09-10T03:53:25.915 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-10T03:53:25.991 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-10T03:53:26.022 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/vg_nvme/lv_21 2024-09-10T03:53:26.053 INFO:teuthology.orchestra.run.smithi050.stdout:connecting to device: nvme2 2024-09-10T03:53:26.055 INFO:tasks.nvme_loop:Connecting nvme_loop smithi050:/dev/vg_nvme/lv_3... 2024-09-10T03:53:26.055 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-10T03:53:26.130 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-10T03:53:26.161 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/vg_nvme/lv_31 2024-09-10T03:53:26.196 INFO:teuthology.orchestra.run.smithi050.stdout:connecting to device: nvme3 2024-09-10T03:53:26.198 INFO:tasks.nvme_loop:Connecting nvme_loop smithi050:/dev/vg_nvme/lv_4... 2024-09-10T03:53:26.198 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-10T03:53:26.273 INFO:teuthology.orchestra.run.smithi050.stdout:1 2024-09-10T03:53:26.304 INFO:teuthology.orchestra.run.smithi050.stdout:/dev/vg_nvme/lv_41 2024-09-10T03:53:26.340 INFO:teuthology.orchestra.run.smithi050.stdout:connecting to device: nvme4 2024-09-10T03:53:26.342 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:53:26.342 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:53:26.398 DEBUG:teuthology.orchestra.run.smithi050:> sudo nvme list -o json 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "Devices":[ 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "NameSpace":1, 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "GenericPath":"/dev/ng4n1", 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "SerialNumber":"e2d852fcc8ac72dd9d09", 2024-09-10T03:53:26.465 INFO:teuthology.orchestra.run.smithi050.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "SectorSize":512 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "NameSpace":1, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "GenericPath":"/dev/ng3n1", 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "SerialNumber":"5996ec7cb7dc2a61b541", 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:26.466 INFO:teuthology.orchestra.run.smithi050.stdout: "SectorSize":512 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "NameSpace":1, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "GenericPath":"/dev/ng2n1", 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "SerialNumber":"fb829e2adb144ce37343", 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: "SectorSize":512 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-10T03:53:26.467 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "NameSpace":1, 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "GenericPath":"/dev/ng1n1", 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "Firmware":"5.14.0-5", 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "ModelNumber":"Linux", 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "SerialNumber":"ea81da236ed925dfb7ac", 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "UsedBytes":95995035648, 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "MaximumLBA":187490304, 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "PhysicalSize":95995035648, 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "SectorSize":512 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: }, 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: { 2024-09-10T03:53:26.468 INFO:teuthology.orchestra.run.smithi050.stdout: "NameSpace":1, 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "GenericPath":"/dev/ng0n1", 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "Firmware":"E2010325", 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "ModelNumber":"INTEL SSDPED1D480GA", 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "SerialNumber":"PHMB7513001F480DGN", 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "UsedBytes":480103981056, 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "MaximumLBA":937703088, 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "PhysicalSize":480103981056, 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: "SectorSize":512 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: } 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-10T03:53:26.469 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-10T03:53:26.470 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T03:53:26.470 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:53:26.470 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/scratch_devs 2024-09-10T03:53:26.536 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T03:53:26.632 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'CEPHADM_AGENT_DOWN'], 'log-only-match': ['CEPHADM_'], 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c'} 2024-09-10T03:53:26.632 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:53:26.633 INFO:tasks.cephadm:Cluster fsid is 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:53:26.633 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T03:53:26.633 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.38', 'mon.c': '[v2:172.21.15.38:3301,v1:172.21.15.38:6790]', 'mon.b': '172.21.15.50'} 2024-09-10T03:53:26.633 INFO:tasks.cephadm:First mon is mon.a on smithi038 2024-09-10T03:53:26.633 INFO:tasks.cephadm:First mgr is y 2024-09-10T03:53:26.633 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T03:53:26.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-09-10T03:53:26.665 DEBUG:teuthology.orchestra.run.smithi050:> sudo hostname $(hostname -s) 2024-09-10T03:53:26.691 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-10T03:53:26.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:53:26.836 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3a5e3f16720f432aea965c40a8f6eb480095ec6c', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82931/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-4885-g3a5e3f16', 'node_name': '172.21.2.4+braggi04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-4885.g3a5e3f16'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-10 02:14:59.834535', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-adk-testing-2024-09-09-1904', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-10T03:53:26.976 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-adk-testing-2024-09-09-1904, sha1 3a5e3f16720f432aea965c40a8f6eb480095ec6c from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:53:26.978 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/x86_64/flavors/default/cephadm 2024-09-10T03:53:26.978 INFO:tasks.cephadm:Downloading cephadm from url: https://3.chacra.ceph.com/binaries/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/x86_64/flavors/default/cephadm 2024-09-10T03:53:26.978 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.367 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 804474 Sep 10 03:53 /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.367 DEBUG:teuthology.orchestra.run.smithi050:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-adk-testing-2024-09-09-1904/3a5e3f16720f432aea965c40a8f6eb480095ec6c/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.754 INFO:teuthology.orchestra.run.smithi050.stdout:-rw-r--r--. 1 ubuntu ubuntu 804474 Sep 10 03:53 /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.755 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.775 DEBUG:teuthology.orchestra.run.smithi050:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T03:53:27.808 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c on all hosts... 2024-09-10T03:53:27.808 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c pull 2024-09-10T03:53:27.818 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c pull 2024-09-10T03:53:29.403 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c... 2024-09-10T03:53:29.485 INFO:teuthology.orchestra.run.smithi050.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c... 2024-09-10T03:54:07.014 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-10T03:54:07.014 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)", 2024-09-10T03:54:07.015 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578", 2024-09-10T03:54:07.015 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-09-10T03:54:07.015 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca" 2024-09-10T03:54:07.015 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-10T03:54:07.015 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-10T03:54:18.336 INFO:teuthology.orchestra.run.smithi050.stdout:{ 2024-09-10T03:54:18.336 INFO:teuthology.orchestra.run.smithi050.stdout: "ceph_version": "ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev)", 2024-09-10T03:54:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "image_id": "d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578", 2024-09-10T03:54:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "repo_digests": [ 2024-09-10T03:54:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca" 2024-09-10T03:54:18.337 INFO:teuthology.orchestra.run.smithi050.stdout: ] 2024-09-10T03:54:18.337 INFO:teuthology.orchestra.run.smithi050.stdout:} 2024-09-10T03:54:18.365 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-09-10T03:54:18.399 DEBUG:teuthology.orchestra.run.smithi050:> sudo mkdir -p /etc/ceph 2024-09-10T03:54:18.427 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-09-10T03:54:18.469 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 777 /etc/ceph 2024-09-10T03:54:18.495 INFO:tasks.cephadm:Writing seed config... 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T03:54:18.497 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T03:54:18.498 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:54:18.498 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T03:54:18.528 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3be9daae-6f28-11ef-bcea-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T03:54:18.528 DEBUG:teuthology.orchestra.run.smithi038:mon.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T03:54:18.570 DEBUG:teuthology.orchestra.run.smithi038:mgr.y> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T03:54:18.619 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T03:54:18.619 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c -v bootstrap --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:54:18.854 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-10T03:54:18.854 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c', '-v', 'bootstrap', '--fsid', '3be9daae-6f28-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-09-10T03:54:18.854 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T03:54:18.854 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-09-10T03:54:18.881 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-10T03:54:18.881 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-09-10T03:54:18.881 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-09-10T03:54:18.889 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T03:54:18.889 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T03:54:18.895 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T03:54:18.895 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:54:18.902 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-10T03:54:18.909 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-10T03:54:18.909 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-10T03:54:18.910 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-09-10T03:54:18.946 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-10T03:54:18.946 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T03:54:18.946 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-09-10T03:54:18.946 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-09-10T03:54:18.952 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T03:54:18.952 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T03:54:18.959 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T03:54:18.959 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:54:18.965 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-10T03:54:18.973 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-10T03:54:18.973 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-10T03:54:18.973 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-09-10T03:54:18.974 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:18.974 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 140119356831440 on /run/cephadm/3be9daae-6f28-11ef-bcea-c7b262605968.lock 2024-09-10T03:54:18.974 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 140119356831440 acquired on /run/cephadm/3be9daae-6f28-11ef-bcea-c7b262605968.lock 2024-09-10T03:54:18.974 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-09-10T03:54:18.975 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-09-10T03:54:18.975 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-10T03:54:18.977 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-09-10T03:54:18.977 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-09-10T03:54:18.979 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T03:54:18.979 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-10T03:54:18.980 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-10T03:54:18.982 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T03:54:18.982 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T03:54:18.982 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T03:54:18.983 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-10T03:54:18.983 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-10T03:54:18.983 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T03:54:18.984 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-10T03:54:18.984 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-10T03:54:18.985 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T03:54:18.985 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T03:54:18.985 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c... 2024-09-10T03:54:19.466 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 2024-09-10T03:54:19.466 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c... 2024-09-10T03:54:19.466 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T03:54:19.466 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T03:54:19.466 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:6e87d7485d6a94ca54378524c236c1e0561edcf2edeb5c2b6e2e509a6fc5a8fa 2024-09-10T03:54:19.467 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 2024-09-10T03:54:19.467 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T03:54:20.404 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev) 2024-09-10T03:54:20.404 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev) 2024-09-10T03:54:20.404 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T03:54:21.326 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-09-10T03:54:21.326 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-09-10T03:54:22.200 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQDtwt9mQvnBJRAAgNiGZDAcuAYHoMccoP32Jw== 2024-09-10T03:54:22.969 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQDuwt9mWhwrHBAA7JlqWZu+BEwJe6Pse4Jrsg== 2024-09-10T03:54:23.858 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQDvwt9ml+2fEBAA8YbUESkoOQ+JaXQpX8r23A== 2024-09-10T03:54:23.858 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-09-10T03:54:24.881 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for a [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:54:24.882 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-09-10T03:54:25.837 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.a on 2024-09-10T03:54:26.227 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T03:54:26.420 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target. 2024-09-10T03:54:26.420 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target. 2024-09-10T03:54:26.686 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a 2024-09-10T03:54:26.688 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T03:54:26.879 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target.wants/ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968@.service. 2024-09-10T03:54:27.395 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:54:27.395 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-10T03:54:27.403 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:54:27.403 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:54:27.403 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-10T03:54:27.403 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T03:54:27.404 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-09-10T03:54:27.404 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-09-10T03:54:28.440 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.545502s) 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T03:54:28.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:28.442 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-09-10T03:54:28.442 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T03:54:29.450 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:29 smithi038 ceph-mon[25019]: from='client.? 172.21.15.38:0/3072809594' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T03:54:29.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T03:54:29.455 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-09-10T03:54:30.484 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-09-10T03:54:30.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:30 smithi038 ceph-mon[25019]: from='client.? 172.21.15.38:0/2611967182' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:54:30.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:30 smithi038 systemd[1]: Stopping Ceph mon.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T03:54:31.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:30 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a[25015]: 2024-09-10T03:54:30.759+0000 7fea2741e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T03:54:31.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:30 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a[25015]: 2024-09-10T03:54:30.759+0000 7fea2741e640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T03:54:31.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:30 smithi038 podman[25267]: 2024-09-10 03:54:30.851476141 +0000 UTC m=+0.198656801 container died 3939492f8fa28875574acc49fa5b0ffc0b338247f67df6d536e5c0b4bb1e911b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T03:54:35.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:34 smithi038 podman[25267]: 2024-09-10 03:54:34.927838156 +0000 UTC m=+4.275018815 container cleanup 3939492f8fa28875574acc49fa5b0ffc0b338247f67df6d536e5c0b4bb1e911b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:54:35.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:34 smithi038 bash[25267]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a 2024-09-10T03:54:37.489 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:37 smithi038 podman[25280]: 2024-09-10 03:54:37.150415628 +0000 UTC m=+6.293459195 container remove 3939492f8fa28875574acc49fa5b0ffc0b338247f67df6d536e5c0b4bb1e911b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:54:37.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:37 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T03:54:37.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:37 smithi038 systemd[1]: Stopped Ceph mon.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:54:37.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:37 smithi038 systemd[1]: Starting Ceph mon.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T03:54:37.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:37 smithi038 podman[25375]: 2024-09-10 03:54:37.391773457 +0000 UTC m=+0.016015644 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c 2024-09-10T03:54:38.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:38 smithi038 podman[25375]: 2024-09-10 03:54:38.425568844 +0000 UTC m=+1.049811068 container create c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T03:54:40.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 podman[25375]: 2024-09-10 03:54:40.280502902 +0000 UTC m=+2.904745083 container init c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 podman[25375]: 2024-09-10 03:54:40.284857244 +0000 UTC m=+2.909099423 container start c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev), process ceph-mon, pid 2 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: pidfile_write: ignore empty --pid-file 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: load: jerasure load: lrc 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: RocksDB version: 7.9.2 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Git sha 0 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Compile date 2024-09-10 00:49:01 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: DB SUMMARY 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: DB Session ID: 9I21KR7DEH88MQCL6Y89 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: CURRENT file: CURRENT 2024-09-10T03:54:40.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 88808 ; 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.error_if_exists: 0 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.create_if_missing: 0 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.paranoid_checks: 1 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.env: 0x560012c8b120 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.info_log: 0x56001502d700 2024-09-10T03:54:40.756 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.statistics: (nil) 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.use_fsync: 0 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_log_file_size: 0 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T03:54:40.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_fallocate: 1 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.use_direct_reads: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.db_log_dir: 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.wal_dir: 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T03:54:40.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.write_buffer_manager: 0x560015031b80 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T03:54:40.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.unordered_write: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.row_cache: None 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.wal_filter: None 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.two_write_queues: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.wal_compression: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.atomic_flush: 0 2024-09-10T03:54:40.761 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.log_readahead_size: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_background_jobs: 2 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_background_compactions: -1 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_subcompactions: 1 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T03:54:40.762 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_open_files: -1 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_background_flushes: -1 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Compression algorithms supported: 2024-09-10T03:54:40.763 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kZSTD supported: 0 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kXpressCompression supported: 0 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kBZip2Compression supported: 0 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kLZ4Compression supported: 1 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kZlibCompression supported: 1 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: kSnappyCompression supported: 1 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T03:54:40.764 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.merge_operator: 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_filter: None 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56001502d320) 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_type: 0 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_index_type: 0 2024-09-10T03:54:40.765 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_shortening: 1 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: checksum: 4 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: no_block_cache: 0 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache: 0x560015021610 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_options: 2024-09-10T03:54:40.766 INFO:journalctl@ceph.mon.a.smithi038.stdout: capacity : 536870912 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_shard_bits : 4 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: strict_capacity_limit : 0 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_compressed: (nil) 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: persistent_cache: (nil) 2024-09-10T03:54:40.767 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size: 4096 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size_deviation: 10 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_restart_interval: 16 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_block_restart_interval: 1 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: metadata_block_size: 4096 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: partition_filters: 0 2024-09-10T03:54:40.768 INFO:journalctl@ceph.mon.a.smithi038.stdout: use_delta_encoding: 1 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: filter_policy: bloomfilter 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: whole_key_filtering: 1 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: verify_compression: 0 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: format_version: 5 2024-09-10T03:54:40.769 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_index_compression: 1 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_align: 0 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout: max_auto_readahead_size: 262144 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout: prepopulate_block_cache: 0 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout: initial_auto_readahead_size: 8192 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T03:54:40.770 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression: NoCompression 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T03:54:40.771 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.num_levels: 7 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T03:54:40.772 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T03:54:40.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:54:40.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T03:54:40.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T03:54:40.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:54:40.773 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:54:40.774 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T03:54:40.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T03:54:40.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:54:40.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T03:54:40.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T03:54:40.775 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T03:54:40.776 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T03:54:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T03:54:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T03:54:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T03:54:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T03:54:40.777 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T03:54:40.778 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T03:54:40.779 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.inplace_update_support: 0 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T03:54:40.780 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.bloom_locality: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.max_successive_merges: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.ttl: 2592000 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T03:54:40.781 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enable_blob_files: false 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.min_blob_size: 0 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b3e3c1a5-5ec6-4246-b3c2-0507ad02fad1 2024-09-10T03:54:40.782 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940480317690, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940480319349, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 85553, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 259, "table_properties": {"data_size": 83700, "index_size": 237, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10145, "raw_average_key_size": 46, "raw_value_size": 77840, "raw_average_value_size": 357, "num_data_blocks": 11, "num_entries": 218, "num_filter_entries": 218, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725940480, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b3e3c1a5-5ec6-4246-b3c2-0507ad02fad1", "db_session_id": "9I21KR7DEH88MQCL6Y89", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940480319520, "job": 1, "event": "recovery_finished"} 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560015050e00 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: DB pointer 0x560015152000 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** DB Stats ** 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:54:40.783 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: L0 2/0 85.41 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Sum 2/0 85.41 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:54:40.784 INFO:journalctl@ceph.mon.a.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 76.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative compaction: 0.00 GB write, 11.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval compaction: 0.00 GB write, 11.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T03:54:40.785 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache BinnedLRUCache@0x560015021610#2 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: starting mon.a rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???) e1 preinit fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).mds e1 new map 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).mds e1 print_map 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: e1 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: btime 2024-09-10T03:54:27:356975+0000 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: legacy client fscid: -1 2024-09-10T03:54:40.786 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout: No filesystems configured 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: monmap epoch 1 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: last_changed 2024-09-10T03:54:24.197396+0000 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: min_mon_release 19 (squid) 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: election_strategy: 1 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:54:40.787 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: fsmap 2024-09-10T03:54:40.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T03:54:40.788 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:40 smithi038 ceph-mon[25393]: mgrmap e1: no daemons active 2024-09-10T03:54:41.239 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T03:54:41.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:41 smithi038 bash[25375]: c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa 2024-09-10T03:54:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:41 smithi038 systemd[1]: Started Ceph mon.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:54:44.085 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T03:54:44.106 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:54:44.107 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-09-10T03:54:44.107 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T03:54:44.107 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T03:54:44.377 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y 2024-09-10T03:54:44.377 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T03:54:44.541 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968.target.wants/ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-3be9daae-6f28-11ef-bcea-c7b262605968@.service. 2024-09-10T03:54:44.883 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:44 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/229746601' entity='client.admin' 2024-09-10T03:54:45.070 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:54:45.070 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-10T03:54:45.079 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:54:45.079 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:54:45.079 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-10T03:54:45.079 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T03:54:45.090 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:54:45.090 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-09-10T03:54:45.099 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-09-10T03:54:45.942 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:45 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1256698364' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "3be9daae-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:54:46.244 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:54:46.245 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:54:46.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:54:27:356975+0000", 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:54:46.247 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:54:27.357660+0000", 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:54:46.248 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-09-10T03:54:49.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:48 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1056697634' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "3be9daae-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:54:49.667 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:54:49.668 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:54:49.669 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:54:27:356975+0000", 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:54:49.670 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:54:27.357660+0000", 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:54:49.671 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-09-10T03:54:50.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: Activating manager daemon y 2024-09-10T03:54:50.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: mgrmap e2: y(active, starting, since 0.00532997s) 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: Manager daemon y is now available 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' 2024-09-10T03:54:50.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:50 smithi038 ceph-mon[25393]: from='mgr.14100 172.21.15.38:0/681447163' entity='mgr.y' 2024-09-10T03:54:52.353 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:52 smithi038 ceph-mon[25393]: mgrmap e3: y(active, since 1.00925s) 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "3be9daae-6f28-11ef-bcea-c7b262605968", 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T03:54:52.964 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T03:54:52.965 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T03:54:52.966 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-10T03:54:27:356975+0000", 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T03:54:52.967 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T03:54:27.357660+0000", 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:54:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-09-10T03:54:53.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:53 smithi038 ceph-mon[25393]: mgrmap e4: y(active, since 2s) 2024-09-10T03:54:53.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:53 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2136350550' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:54.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T03:54:54.142 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-09-10T03:54:54.344 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:54 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3685198023' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-10T03:54:54.344 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:54 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3685198023' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T03:54:55.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:55 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1530991100' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-10T03:54:55.504 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:54:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ignoring --setgroup ceph since I am not root 2024-09-10T03:54:56.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:56 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1530991100' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T03:54:56.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:56 smithi038 ceph-mon[25393]: mgrmap e5: y(active, since 5s) 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:54:57.102 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-10T03:54:57.103 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-09-10T03:54:57.418 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:54:57 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3591204244' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T03:55:01.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: Active manager daemon y restarted 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: Activating manager daemon y 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: mgrmap e6: y(active, starting, since 0.0125904s) 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T03:55:01.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:00 smithi038 ceph-mon[25393]: Manager daemon y is now available 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-09-10T03:55:02.199 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying orchestrator module is enabled... 2024-09-10T03:55:03.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:02 smithi038 ceph-mon[25393]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T03:55:03.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:02 smithi038 ceph-mon[25393]: mgrmap e7: y(active, since 1.00998s) 2024-09-10T03:55:03.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:02 smithi038 ceph-mon[25393]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T03:55:03.993 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:03 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1745024793' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-10T03:55:04.246 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-10T03:55:04.246 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-09-10T03:55:04.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1745024793' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-10T03:55:04.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: mgrmap e8: y(active, since 3s) 2024-09-10T03:55:04.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:04.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: Found migration_current of "None". Setting to last migration. 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T03:55:04.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:04 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T03:55:05.852 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-mon[25393]: mgrmap e9: y(active, since 4s) 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-mon[25393]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T03:55:05.652+0000 7f200ad51640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:03:55:05] ENGINE Error in HTTPServer.serve 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Traceback (most recent call last): 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._connections.run(self.expiration_interval) 2024-09-10T03:55:05.853 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._run(expiration_interval) 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s = self.context.wrap_socket( 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:55:05.854 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: return self.sslsocket_class._create( 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self.do_handshake() 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._sslobj.do_handshake() 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:55:05.855 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:05 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:05] ENGINE Bus STARTING 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:05] ENGINE Serving on https://172.21.15.38:7150 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:05] ENGINE Error in HTTPServer.serve 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:55:06.723 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-10T03:55:06.724 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:05] ENGINE Serving on http://172.21.15.38:8765 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:05] ENGINE Bus STARTED 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:06.725 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:06 smithi038 ceph-mon[25393]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:06.725 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T03:55:06.726 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-09-10T03:55:07.942 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Generating public/private rsa key pair. 2024-09-10T03:55:07.942 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Your identification has been saved in /tmp/tmpyi4b08o_/key 2024-09-10T03:55:07.942 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Your public key has been saved in /tmp/tmpyi4b08o_/key.pub 2024-09-10T03:55:07.942 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: The key fingerprint is: 2024-09-10T03:55:07.942 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: SHA256:LWxtPshjXCAYTaae7UF/gChVmRTnv/HwalZKSWW6rg0 ceph-3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: The key's randomart image is: 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: +---[RSA 3072]----+ 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | o=*+. | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | . *++ o | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | . + + + + | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | o + + *o | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | o o S.Xo | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | . = B+*. | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | . Eo+oo | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | . +=o | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: | .+o | 2024-09-10T03:55:07.943 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: +----[SHA256]-----+ 2024-09-10T03:55:09.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:08 smithi038 ceph-mon[25393]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:09.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:08 smithi038 ceph-mon[25393]: Generating ssh key... 2024-09-10T03:55:09.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:08 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:09.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:08 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:09.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+3bBiPAqdKZ4pdQ58ldoCVm8iRYMPQYw6zTMzuZcFscl7VRkJiz8balyYWAtRkMxHZPC0J++LTXwTbKe728VULkXcCFNa/7tFYmZGRD/hiQf6sl5zOWUGfcN5rNYu5rVDbdMlZC3pOcaHnjKtUY3NpA2E9ozV5rIJxbKetIwTSG9gqlHFGtiytu/HesKIH59ix24nadtwNANcKMNPw/zBhLZHaiQQZxAGm7E3g/AW35iMaXZ517DyVBEVO8ydQVVy6rHklgB42GmURXRFRMckgsIeNvFTvNAC7saAVWyGkh4Ke5HZE+1IXkfGpHBRIvOK8wC6GF4xAW4NGi10eGctWOgjlpxOWQHkWQfPiD/E35Iw/ggov2W6Wh7+qQN/XU+nL1aujsHm8L5GKLgUv6UFM7eWrrb9ctrcA3dgnjkkra+6jjMXHWIBSIpL+ChgU4AtvG/ozKJrKP0ofWCCFU4DYlrM6vGgcu56s1kWcILpksh0qpBlvOtON8jis78Lcr0= ceph-3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:09.506 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T03:55:09.506 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T03:55:09.513 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-09-10T03:55:09.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:09 smithi038 ceph-mon[25393]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:10.837 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:10 smithi038 ceph-mon[25393]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:11.707 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:11 smithi038 ceph-mon[25393]: Deploying cephadm binary to smithi038 2024-09-10T03:55:14.665 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-09-10T03:55:14.665 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mon service... 2024-09-10T03:55:15.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:14 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:15.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:14 smithi038 ceph-mon[25393]: Added host smithi038 2024-09-10T03:55:15.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:14 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:16 smithi038 ceph-mon[25393]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:16 smithi038 ceph-mon[25393]: Saving service mon spec with placement count:5 2024-09-10T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:16 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:16.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:16 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:16.787 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T03:55:16.787 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mgr service... 2024-09-10T03:55:17.997 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:17 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:17.997 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:17 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:18.531 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: Saving service mgr spec with placement count:2 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.agent.smithi038", "caps": []} : dispatch 2024-09-10T03:55:18.789 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:18 smithi038 ceph-mon[25393]: from='mgr.14118 172.21.15.38:0/1337052038' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi038", "caps": []}]': finished 2024-09-10T03:55:20.446 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:20 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1635540947' entity='client.admin' 2024-09-10T03:55:20.984 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-09-10T03:55:21.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:21 smithi038 ceph-mon[25393]: Deploying daemon agent.smithi038 on smithi038 2024-09-10T03:55:21.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:21 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2810494536' entity='client.admin' 2024-09-10T03:55:22.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:22 smithi038 ceph-mon[25393]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:22.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:22 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3134020521' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-10T03:55:22.504 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:22 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ignoring --setuser ceph since I am not root 2024-09-10T03:55:22.505 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:22 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ignoring --setgroup ceph since I am not root 2024-09-10T03:55:23.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:23 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3134020521' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T03:55:23.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:23 smithi038 ceph-mon[25393]: mgrmap e10: y(active, since 21s) 2024-09-10T03:55:24.273 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 10, 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-10T03:55:24.274 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 10... 2024-09-10T03:55:24.485 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:24 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1723248932' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-10T03:55:28.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: Active manager daemon y restarted 2024-09-10T03:55:28.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: Activating manager daemon y 2024-09-10T03:55:28.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: mgrmap e11: y(active, starting, since 0.00834636s) 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T03:55:28.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:27 smithi038 ceph-mon[25393]: Manager daemon y is now available 2024-09-10T03:55:29.067 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:29.067 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:29.067 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T03:55:29.067 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T03:55:29.067 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:28 smithi038 ceph-mon[25393]: mgrmap e12: y(active, since 1.00761s) 2024-09-10T03:55:29.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-10T03:55:29.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 12, 2024-09-10T03:55:29.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T03:55:29.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-10T03:55:29.069 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 10 is available 2024-09-10T03:55:29.070 INFO:teuthology.orchestra.run.smithi038.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-10T03:55:29.320 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T03:55:29.102+0000 7f0ec4229640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:03:55:29] ENGINE Error in HTTPServer.serve 2024-09-10T03:55:29.320 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Traceback (most recent call last): 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._connections.run(self.expiration_interval) 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._run(expiration_interval) 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s = self.context.wrap_socket( 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: return self.sslsocket_class._create( 2024-09-10T03:55:29.321 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:55:29.322 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self.do_handshake() 2024-09-10T03:55:29.322 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:55:29.322 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._sslobj.do_handshake() 2024-09-10T03:55:29.322 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:55:29.322 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T03:55:30.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:28] ENGINE Bus STARTING 2024-09-10T03:55:30.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:28] ENGINE Serving on http://172.21.15.38:8765 2024-09-10T03:55:30.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:29] ENGINE Serving on https://172.21.15.38:7150 2024-09-10T03:55:30.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:29] ENGINE Bus STARTED 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:29 smithi038 ceph-mon[25393]: [10/Sep/2024:03:55:29] ENGINE Error in HTTPServer.serve 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-10T03:55:30.005 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-10T03:55:30.006 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T03:55:31.109 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:31.109 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: mgrmap e13: y(active, since 2s) 2024-09-10T03:55:31.109 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:31.110 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T03:55:31.110 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:31.110 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:30 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.agent.smithi038", "caps": []} : dispatch 2024-09-10T03:55:31.112 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUPRkdygG3K6hrZUOUdUXTWJ9x9CkwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkxMDAzNTUzMFoXDTM0\nMDkxMTAzNTUzMFowFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjM4MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAwe7/JmN8nqKcVKsY2IQXjC0lNGpwcmbhxvo/\nLeIaGTD7VxPJR7iQfPmA4ENFX6kBRMe0X7pRjYyzDH4u0u9VRNbMzTtxwphOiJkV\nEuvmvsV4bxeGZ68eE6CuBvxcDw7XaObnUUsEm0hcxq0WcOFzJCHBDWyec4Zlny20\nUjU3R8oRXKMO60Uzp+dzOo1cHbLtdHpCgGZ0ydtkAXeR/73gkQT9f1lcWX28bQNh\nyo0arxj+ov6uEWTiYbrOKcM45dUYlcfJTPdP4ihe25vYmCiO8+uX+batzvoEjWD0\ncYb717waNV8S+OIP9LuwFgS0JlwSF6vY1GoMidYr44+K3j/zjA5OU7GDVAdlv2w9\n/JXFG/7x3M6vYbHsPgm6UJBhPjtM0jmbjCy6ybkovkyxyMkbmmdcszAvnnNMXw8x\na6rSolwc4bz+rI6b4+Q9zk1bar9b9KJgwYiNCLP4oWFql4xS14ZvXHgxN4Nkb9fM\n6lT1sUJK0O90zMxzMmKG3og4fDnFsY4miJxBslXrqKVso4rnY2r4VDOeixR6sdmD\nwHND7QoMoTiiRmoaBAYjkHLsExrC4k/l5k8oUvpnqK33+eYWmvAAAa2EMIEKE+Ks\nIpViY2EbTj7kHnkejnq4wiPwBLPhdg0TcursKd7evt3IY9RGfe7besvKf3fDiupS\nUOyV6YsCAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDM4ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPJjAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQAH\nsfMURh79ElLi0sgi7V+KuyzjyBAlzzqPF/KqztslpqMetANZHD8PKFj140lcMqUc\nTZDiu2qU17nPfqMBirtKPPkpFPvTemZpTYxwK4Mua5qnY4ou92rO3Lc2spxfnE5R\nZcitL1G6xQiK2u/s/llYrywphBnh3ZpBltk5duLWR0GVKgjbpAoJsy9Jcr4OZpw5\nyyXJ/fREq2vf5VfS/wrEBKuEND752ikyQsBJD5rqQL6mcMueFS+wn2/Gq53aTzXJ\n1AuYffZwjeHeUsaTDASYDh2w1J67I4q9q/PLDKeix9lYLWoJn1maPP8b6J8JF69/\n4YXxm3cW0FVU2dQAftLN6RC/5CC15khYlyjeTJc9N4TfPCYGNjzQiuK7HfM6n7PQ\n7YEB0Q2vei8FEPqvOTTyidFpLfMXH1iRPIga2TUB1uMyRaqaoG+g4ryYGUnaKyam\niCNBLeAaIhWzGkr7unPd4ptbl0vljzTOhQaPFpNhQJknz5JaeIfEYdd5mQJMn+qg\nDNfu6au9+Uxa7umWwjnACAt/sNeJ2csQTHzWcVYXe5bovlh6h8+6N0in8zJwG4FU\nPfZoBZc7OqHIlw43XL51fxzoY2zSr6rG1EBHXnseLVU1ejIMykUloiMSfuq8PaBw\njE7RG2E+xHk+urP8119N7FCH9v9/CfbxOhT+CLv81A==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKAIBAAKCAgEAwe7/JmN8nqKcVKsY2IQXjC0lNGpwcmbhxvo/LeIaGTD7VxPJ\nR7iQfPmA4ENFX6kBRMe0X7pRjYyzDH4u0u9VRNbMzTtxwphOiJkVEuvmvsV4bxeG\nZ68eE6CuBvxcDw7XaObnUUsEm0hcxq0WcOFzJCHBDWyec4Zlny20UjU3R8oRXKMO\n60Uzp+dzOo1cHbLtdHpCgGZ0ydtkAXeR/73gkQT9f1lcWX28bQNhyo0arxj+ov6u\nEWTiYbrOKcM45dUYlcfJTPdP4ihe25vYmCiO8+uX+batzvoEjWD0cYb717waNV8S\n+OIP9LuwFgS0JlwSF6vY1GoMidYr44+K3j/zjA5OU7GDVAdlv2w9/JXFG/7x3M6v\nYbHsPgm6UJBhPjtM0jmbjCy6ybkovkyxyMkbmmdcszAvnnNMXw8xa6rSolwc4bz+\nrI6b4+Q9zk1bar9b9KJgwYiNCLP4oWFql4xS14ZvXHgxN4Nkb9fM6lT1sUJK0O90\nzMxzMmKG3og4fDnFsY4miJxBslXrqKVso4rnY2r4VDOeixR6sdmDwHND7QoMoTii\nRmoaBAYjkHLsExrC4k/l5k8oUvpnqK33+eYWmvAAAa2EMIEKE+KsIpViY2EbTj7k\nHnkejnq4wiPwBLPhdg0TcursKd7evt3IY9RGfe7besvKf3fDiupSUOyV6YsCAwEA\nAQKCAgBPPRE4hLbC9GsOtzf4I4ShjEKi14ipXbsLe208GKGUQGy5qAZ6TQGQZruP\nJj1euCWiDjz3TiALR31S8e281vhOn7tfcdXp7iRMcMvpBsUplnK+ZWzPyX/vAhyJ\nEyykJGrEXHAQSagQsHDj5o4AQkWy8i24h/R3/GZiMyD14/N53Cja4m7ylzxjctCk\nkGSeZ+DY0imA4BwA7CjA5rObPGBYsKLJGWB/2j2Whya6RYwtFffY7ll+FuvehZ69\n9pu1YKviaxanYXdczyTu8w79ILlEFbNrXwZGRuZ8Q6ytBDu7aoqE2xtb4LYnR1bL\nsFFRSRZFYLxAEjKcUQtut9REOfoOvM4gsqBKDlwLFH2jLcXMBb6xTKPUO4z+KEye\nIq6yOTRTKPJwvbEJbb2q6HuurD8a/DOQAeqd2gWof+tlps67eXzMPaxbRxgiSlgu\nPOA+U/ej691FK2fEvCWEXa6FUOy6yw9B2KIESFGeUXzO+MtbCnJJk3uix8Kmu/1n\nyl0V2hg3QtaIapcF4fAm+MwFZ6jRZIRROEDNs33bj/F/ShqhIy7OSjfeCA+buPdI\ndCJfyTrL33EjwESwVbNTVG3Eiep8TnB0S27tQ/2aZH313A6o3vVVp4Ill2V6UWdC\n/lZeFm7CZDi6pkPF88+Ju4JPdbA/av96YhPs5aT0ZJJ5PwDR9QKCAQEA+5s28i+H\nRRF0ZwH/6x2673oqWb6oiZjHYUsml6DH8axkPtQKpejZOsC/8R1MWTxHvuVlyAop\n6/P8pbyfotx6VE1sm5lgkMH9vXafMb3URY+tvUP9b2vi0BPFfW0/ubk/SnVZiS52\nGUuMzb3ZS8bw8WYhfSiLVw067yGst5GyGADOEnWR+87MX6kc+p//brA4NMmP+FEm\nkQxu8i6flU0cOnsvwNvkJEWsano3bgFqry0QwBWTwFBfYYTLCPDbJRDlcq1/nN/G\njPnx0f3rxgslsjvhsTwoQqMGGp3UYlj/jyThBVz9rHIYeZ4uloK+fiMUkBr6QRVA\nZ8R4Qv0zyoo1bwKCAQEAxVH1+mLROOnx+wgMmYMzRWjIXGhPHvlTzGW22VCB6fE6\nv5JW64jphY011GlOP0x8C9qlV6Oq0LmXziaVcxhROT03fazbG/eDV3Cpryx1kAL0\nAea57dw9td5TCwU+oImGzGJJQo8pxWYQyfczIa61GBjvzvHCr4jxYO7eiawqSFWl\nkMrw47EH3O85VHN9Weam2Llr7D0bbpHUbVfNk7mDT/QXUbHk0ts7gn1BrblQN1yK\nxUAwqbaYRODsvh4n6TVpnEVq9rU5RL0TGP/WFqt0BvzdIbo2G8LNbPGhHsP+Bx2Z\n9WiP6tDQ0x5toa2kiI6jdfI3dTVzIb3cqDG1h1OXpQKCAQBslC0AEtazTbcIBOrm\nEexsEDJajFa7Gn1gVUgQQ0hF2Plhy6vUsUmdezTZ3qTmFYd4sBJeY1R4CNDMzTuu\n0qrP2qG1/FiZ372al6hXlO4YHssBVGf4P3ygAwhE+nJ3uaPAhCfhdXVtsRbQas9t\nTisIYeSt2GNZa9I1BDk+9tb+WMqVWLtpCXNVDOv+lDFtGQsIbLt+k7aq/cybIten\nXKGD80lGVx57ML4ZLSjihiX99tq1qhoa8hRXRzE/aIGJ22LJPGpHzqImkG51CrlG\nu+TGp+Tb0XQmXafysI3I61NN+MJKyDiIxKwfP84nEcEXOhhTn/DDSWKD5VtovVAu\nZbptAoIBAQCTLY4uw9RRAlamqptTScRNalnAa2FQtiImHFwlfeJZD8CFMe85Yfbk\n0a4sEJv3pOIFqYGYfBCl/TjY4Hiqz6GN8zZXvD32f0jtSLCYY64Qy9MbM/DNFXDG\nHd2NI81RyyLLgigoyEYc4gN10bfG0Nxs6/S7RCK18YwxhBpjUBJ9v7xt4qGJA/vc\ndGcrsbT3AI7EqpFwNSgn8ggjwtEo9b+XPHbETc9i43lg3HLH1KibkqBDfpyNig/L\nmIWdTTrxvRBYKoMdH5FmzyzWgcU7RYmWyLdcN9uyGzxEnXpTS1dIEXFL1kE6oaSR\nRGn7pVxLKzGLgS6Sc1q7+TUEE4fWN6itAoIBAFr1qG53pm+C1Ie9MJhfIgh4polr\nUe4BaUJYy/cby9Ok+RvbMW7zyzvhyVDoYvbCg3I12MyeDzoT6pvTV3ZKI4ZMusy0\nMctCzVrdbU9ZqW0xBCwRV7RQfmf2D2G5Fmj7M7H/vFykKtIwxdN5iA6WeAA3cHpL\nFQhOSv6tfZskrNgRoRG8FhiZLrsU0Jl4YwoF8HUtgqcYKXDlKwzzY2AOcH+RUift\n5SkoPDp/Z587udEXD7f0V3PuRzZ/TbZT4b+2+Bsf1SAFBKmLaiDyuTBNiArjQaOu\nJ2vTeQtv8LdzrREYFMafoiUHyfKUmCJdOd+tJfrDUCb8pMszk6pm6j4RxAo=\n-----END RSA PRIVATE KEY-----\n"} 2024-09-10T03:55:32.301 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-10T03:55:32.929 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:32 smithi038 ceph-mon[25393]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:32.929 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:32 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:32.929 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:32 smithi038 ceph-mon[25393]: Deploying daemon agent.smithi038 on smithi038 2024-09-10T03:55:33.508 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-10T03:55:33.509 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-09-10T03:55:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:33 smithi038 ceph-mon[25393]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:33 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:34.986 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:34 smithi038 ceph-mon[25393]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:34.987 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:34 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:34.989 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$HE9XQyu6QsGq1BTXrJD4y.5lWHb2baELwhlevUnF9bDFQ.53FbOC6", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725940534, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T03:55:34.990 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-09-10T03:55:36.128 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T03:55:36.139 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T03:55:36.139 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-10T03:55:36.147 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T03:55:36.147 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-10T03:55:36.147 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-10T03:55:36.147 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: Password: jnfnf9gxny 2024-09-10T03:55:36.150 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:36.151 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config directory 2024-09-10T03:55:36.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:35 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1940334752' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-10T03:55:37.510 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T03:55:37.510 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T03:55:37.510 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.510 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:37.511 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-09-10T03:55:37.555 INFO:tasks.cephadm:Fetching config... 2024-09-10T03:55:37.556 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:55:37.556 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T03:55:37.572 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T03:55:37.572 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:55:37.572 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T03:55:37.627 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T03:55:37.628 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:55:37.628 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T03:55:37.810 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T03:55:37.810 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:55:37.810 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T03:55:37.868 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T03:55:37.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+3bBiPAqdKZ4pdQ58ldoCVm8iRYMPQYw6zTMzuZcFscl7VRkJiz8balyYWAtRkMxHZPC0J++LTXwTbKe728VULkXcCFNa/7tFYmZGRD/hiQf6sl5zOWUGfcN5rNYu5rVDbdMlZC3pOcaHnjKtUY3NpA2E9ozV5rIJxbKetIwTSG9gqlHFGtiytu/HesKIH59ix24nadtwNANcKMNPw/zBhLZHaiQQZxAGm7E3g/AW35iMaXZ517DyVBEVO8ydQVVy6rHklgB42GmURXRFRMckgsIeNvFTvNAC7saAVWyGkh4Ke5HZE+1IXkfGpHBRIvOK8wC6GF4xAW4NGi10eGctWOgjlpxOWQHkWQfPiD/E35Iw/ggov2W6Wh7+qQN/XU+nL1aujsHm8L5GKLgUv6UFM7eWrrb9ctrcA3dgnjkkra+6jjMXHWIBSIpL+ChgU4AtvG/ozKJrKP0ofWCCFU4DYlrM6vGgcu56s1kWcILpksh0qpBlvOtON8jis78Lcr0= ceph-3be9daae-6f28-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T03:55:38.060 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:37 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3008167980' entity='client.admin' 2024-09-10T03:55:38.064 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+3bBiPAqdKZ4pdQ58ldoCVm8iRYMPQYw6zTMzuZcFscl7VRkJiz8balyYWAtRkMxHZPC0J++LTXwTbKe728VULkXcCFNa/7tFYmZGRD/hiQf6sl5zOWUGfcN5rNYu5rVDbdMlZC3pOcaHnjKtUY3NpA2E9ozV5rIJxbKetIwTSG9gqlHFGtiytu/HesKIH59ix24nadtwNANcKMNPw/zBhLZHaiQQZxAGm7E3g/AW35iMaXZ517DyVBEVO8ydQVVy6rHklgB42GmURXRFRMckgsIeNvFTvNAC7saAVWyGkh4Ke5HZE+1IXkfGpHBRIvOK8wC6GF4xAW4NGi10eGctWOgjlpxOWQHkWQfPiD/E35Iw/ggov2W6Wh7+qQN/XU+nL1aujsHm8L5GKLgUv6UFM7eWrrb9ctrcA3dgnjkkra+6jjMXHWIBSIpL+ChgU4AtvG/ozKJrKP0ofWCCFU4DYlrM6vGgcu56s1kWcILpksh0qpBlvOtON8jis78Lcr0= ceph-3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:38.077 DEBUG:teuthology.orchestra.run.smithi050:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+3bBiPAqdKZ4pdQ58ldoCVm8iRYMPQYw6zTMzuZcFscl7VRkJiz8balyYWAtRkMxHZPC0J++LTXwTbKe728VULkXcCFNa/7tFYmZGRD/hiQf6sl5zOWUGfcN5rNYu5rVDbdMlZC3pOcaHnjKtUY3NpA2E9ozV5rIJxbKetIwTSG9gqlHFGtiytu/HesKIH59ix24nadtwNANcKMNPw/zBhLZHaiQQZxAGm7E3g/AW35iMaXZ517DyVBEVO8ydQVVy6rHklgB42GmURXRFRMckgsIeNvFTvNAC7saAVWyGkh4Ke5HZE+1IXkfGpHBRIvOK8wC6GF4xAW4NGi10eGctWOgjlpxOWQHkWQfPiD/E35Iw/ggov2W6Wh7+qQN/XU+nL1aujsHm8L5GKLgUv6UFM7eWrrb9ctrcA3dgnjkkra+6jjMXHWIBSIpL+ChgU4AtvG/ozKJrKP0ofWCCFU4DYlrM6vGgcu56s1kWcILpksh0qpBlvOtON8jis78Lcr0= ceph-3be9daae-6f28-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T03:55:38.133 INFO:teuthology.orchestra.run.smithi050.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+3bBiPAqdKZ4pdQ58ldoCVm8iRYMPQYw6zTMzuZcFscl7VRkJiz8balyYWAtRkMxHZPC0J++LTXwTbKe728VULkXcCFNa/7tFYmZGRD/hiQf6sl5zOWUGfcN5rNYu5rVDbdMlZC3pOcaHnjKtUY3NpA2E9ozV5rIJxbKetIwTSG9gqlHFGtiytu/HesKIH59ix24nadtwNANcKMNPw/zBhLZHaiQQZxAGm7E3g/AW35iMaXZ517DyVBEVO8ydQVVy6rHklgB42GmURXRFRMckgsIeNvFTvNAC7saAVWyGkh4Ke5HZE+1IXkfGpHBRIvOK8wC6GF4xAW4NGi10eGctWOgjlpxOWQHkWQfPiD/E35Iw/ggov2W6Wh7+qQN/XU+nL1aujsHm8L5GKLgUv6UFM7eWrrb9ctrcA3dgnjkkra+6jjMXHWIBSIpL+ChgU4AtvG/ozKJrKP0ofWCCFU4DYlrM6vGgcu56s1kWcILpksh0qpBlvOtON8jis78Lcr0= ceph-3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:55:38.146 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T03:55:38.636 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.924 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2569040905' entity='client.admin' 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:39.925 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:40.383 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T03:55:40.383 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T03:55:41.434 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:55:41.451 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:41 smithi038 ceph-mon[25393]: mgrmap e14: y(active, since 12s) 2024-09-10T03:55:44.107 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.107 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.108 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:44.698 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi050 2024-09-10T03:55:44.698 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:55:44.698 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.conf 2024-09-10T03:55:44.719 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:55:44.719 DEBUG:teuthology.orchestra.run.smithi050:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:44.778 INFO:tasks.cephadm:Adding host smithi050 to orchestrator... 2024-09-10T03:55:44.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch host add smithi050 2024-09-10T03:55:45.033 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:45.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:46.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:46 smithi038 ceph-mon[25393]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi050", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:47.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:47 smithi038 ceph-mon[25393]: Deploying cephadm binary to smithi050 2024-09-10T03:55:48.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:48 smithi038 ceph-mon[25393]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:49.421 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi050' with addr '172.21.15.50' 2024-09-10T03:55:50.038 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T03:55:50.291 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:55:50.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: Added host smithi050 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:55:50.697 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T03:55:50.698 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:55:50.698 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:50 smithi038 ceph-mon[25393]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:55:50.981 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:55:50.981 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.50", "hostname": "smithi050", "labels": [], "status": ""}] 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.agent.smithi050", "caps": []} : dispatch 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi050", "caps": []}]': finished 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T03:55:51.557 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:51 smithi038 ceph-mon[25393]: Deploying daemon agent.smithi050 on smithi050 2024-09-10T03:55:51.587 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T03:55:51.588 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T03:55:51.836 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:55:53.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:53.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:53.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:52 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3887315603' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T03:55:53.479 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-09-10T03:55:54.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:53 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:54.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:53 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3887315603' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T03:55:54.022 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:53 smithi038 ceph-mon[25393]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:55:54.058 INFO:tasks.cephadm:Adding mon.a on smithi038 2024-09-10T03:55:54.059 INFO:tasks.cephadm:Adding mon.c on smithi038 2024-09-10T03:55:54.059 INFO:tasks.cephadm:Adding mon.b on smithi050 2024-09-10T03:55:54.059 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b' 2024-09-10T03:55:54.948 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:55:55.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:54 smithi038 ceph-mon[25393]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:55.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:54 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:55.578 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled mon update... 2024-09-10T03:55:56.239 DEBUG:teuthology.orchestra.run.smithi038:mon.c> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T03:55:56.242 DEBUG:teuthology.orchestra.run.smithi050:mon.b> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T03:55:56.244 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T03:55:56.244 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:55:56.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:55:57.119 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:55:57.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:56 smithi038 ceph-mon[25393]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:57.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:56 smithi038 ceph-mon[25393]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:55:57.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:56 smithi038 ceph-mon[25393]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b;count:3 2024-09-10T03:55:57.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:56 smithi038 ceph-mon[25393]: Deploying daemon mon.b on smithi050 2024-09-10T03:55:57.907 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:55:57.907 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":1,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:54:24.197396Z","created":"2024-09-10T03:54:24.197396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:55:57.907 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 1 2024-09-10T03:55:58.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:57 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1981209828' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:55:59.232 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:55:58 smithi038 ceph-mon[25393]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:55:59.483 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T03:55:59.483 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:55:59.739 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:56:00.526 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:56:00.527 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":1,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:54:24.197396Z","created":"2024-09-10T03:54:24.197396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T03:56:00.527 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 1 2024-09-10T03:56:01.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:00 smithi038 ceph-mon[25393]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:01.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:01.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:01.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:01.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:00 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/528411821' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:01.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:01 smithi050 systemd[1]: Started Ceph mon.b for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:56:02.101 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T03:56:02.101 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:56:02.380 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:56:06.650 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:56:06.651 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:56:01.265123Z","created":"2024-09-10T03:54:24.197396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T03:56:06.651 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 2 2024-09-10T03:56:06.651 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:06.651 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.651 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: mon.a calling monitor election 2024-09-10T03:56:06.651 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: mon.b calling monitor election 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: monmap epoch 2 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: last_changed 2024-09-10T03:56:01.265123+0000 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:06.652 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: min_mon_release 19 (squid) 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: election_strategy: 1 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: fsmap 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: mgrmap e14: y(active, since 38s) 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: overall HEALTH_OK 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:06.653 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:06.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:06.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: mon.a calling monitor election 2024-09-10T03:56:06.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: mon.b calling monitor election 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: monmap epoch 2 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: last_changed 2024-09-10T03:56:01.265123+0000 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:06.720 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: min_mon_release 19 (squid) 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: election_strategy: 1 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: fsmap 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: mgrmap e14: y(active, since 38s) 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: overall HEALTH_OK 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:06.721 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:07.683 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:07 smithi038 ceph-mon[25393]: Deploying daemon mon.c on smithi038 2024-09-10T03:56:07.683 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:07 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/704816649' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:07.683 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:07 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:07.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:07 smithi050 ceph-mon[26687]: Deploying daemon mon.c on smithi038 2024-09-10T03:56:07.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:07 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/704816649' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:07.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:07 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:08.228 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T03:56:08.228 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:56:08.494 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:56:08.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:08 smithi050 ceph-mon[26687]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:08.672 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:08 smithi038 ceph-mon[25393]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:09.170 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:56:09.170 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":2,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:56:01.265123Z","created":"2024-09-10T03:54:24.197396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T03:56:09.170 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 2 2024-09-10T03:56:09.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 systemd[1]: Starting Ceph mon.c for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T03:56:09.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:09 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/843627315' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:09.626 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/843627315' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 podman[30748]: 2024-09-10 03:56:09.269049345 +0000 UTC m=+0.016511784 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 podman[30748]: 2024-09-10 03:56:09.38217617 +0000 UTC m=+0.129638645 container create 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 podman[30748]: 2024-09-10 03:56:09.601504584 +0000 UTC m=+0.348967014 container init 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True) 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 podman[30748]: 2024-09-10 03:56:09.60514048 +0000 UTC m=+0.352602910 container start 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: ceph version 19.3.0-4885-g3a5e3f16 (3a5e3f16720f432aea965c40a8f6eb480095ec6c) squid (dev), process ceph-mon, pid 2 2024-09-10T03:56:09.627 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pidfile_write: ignore empty --pid-file 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: load: jerasure load: lrc 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: RocksDB version: 7.9.2 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Git sha 0 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Compile date 2024-09-10 00:49:01 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: DB SUMMARY 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: DB Session ID: HIWPTGX1XFPLSA515AOT 2024-09-10T03:56:10.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: CURRENT file: CURRENT 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.error_if_exists: 0 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.create_if_missing: 0 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.paranoid_checks: 1 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.env: 0x55e5ac8a7120 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T03:56:10.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.info_log: 0x55e5ad5cd700 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.statistics: (nil) 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.use_fsync: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_log_file_size: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_fallocate: 1 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T03:56:10.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.use_direct_reads: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.db_log_dir: 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.wal_dir: 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T03:56:10.008 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.write_buffer_manager: 0x55e5ad5d1b80 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.unordered_write: 0 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T03:56:10.009 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.row_cache: None 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.wal_filter: None 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.two_write_queues: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.wal_compression: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.atomic_flush: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.log_readahead_size: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T03:56:10.010 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_background_jobs: 2 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_background_compactions: -1 2024-09-10T03:56:10.011 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_subcompactions: 1 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_open_files: -1 2024-09-10T03:56:10.013 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_background_flushes: -1 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Compression algorithms supported: 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kZSTD supported: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kXpressCompression supported: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kBZip2Compression supported: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kLZ4Compression supported: 1 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kZlibCompression supported: 1 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: kSnappyCompression supported: 1 2024-09-10T03:56:10.014 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.merge_operator: 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_filter: None 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e5ad5cd320) 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T03:56:10.015 INFO:journalctl@ceph.mon.c.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_type: 0 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: data_block_index_type: 0 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_shortening: 1 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: checksum: 4 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: no_block_cache: 0 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache: 0x55e5ad5c1610 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_options: 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: capacity : 536870912 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: num_shard_bits : 4 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: strict_capacity_limit : 0 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-09-10T03:56:10.016 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_cache_compressed: (nil) 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: persistent_cache: (nil) 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_size: 4096 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_size_deviation: 10 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_restart_interval: 16 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: index_block_restart_interval: 1 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: metadata_block_size: 4096 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: partition_filters: 0 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: use_delta_encoding: 1 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: filter_policy: bloomfilter 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: whole_key_filtering: 1 2024-09-10T03:56:10.017 INFO:journalctl@ceph.mon.c.smithi038.stdout: verify_compression: 0 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: format_version: 5 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: enable_index_compression: 1 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: block_align: 0 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: max_auto_readahead_size: 262144 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: prepopulate_block_cache: 0 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: initial_auto_readahead_size: 8192 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T03:56:10.018 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression: NoCompression 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.num_levels: 7 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T03:56:10.019 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T03:56:10.020 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T03:56:10.021 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T03:56:10.022 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.inplace_update_support: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.bloom_locality: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.max_successive_merges: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T03:56:10.023 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.ttl: 2592000 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enable_blob_files: false 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.min_blob_size: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T03:56:10.024 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 859bdd43-4ce8-41a0-9052-8b2baf39c77a 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940569632792, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940569633383, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725940569, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "859bdd43-4ce8-41a0-9052-8b2baf39c77a", "db_session_id": "HIWPTGX1XFPLSA515AOT", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725940569633520, "job": 1, "event": "recovery_finished"} 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e5ad5f0e00 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: DB pointer 0x55e5ad6f2000 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** DB Stats ** 2024-09-10T03:56:10.025 INFO:journalctl@ceph.mon.c.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.026 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:56:10.027 INFO:journalctl@ceph.mon.c.smithi038.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: Block cache BinnedLRUCache@0x55e5ad5c1610#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: using public_addrv [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: starting mon.c rank -1 at public addrs [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] at bind addrs [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(???) e0 preinit fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: e1 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: btime 2024-09-10T03:54:27:356975+0000 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T03:56:10.028 INFO:journalctl@ceph.mon.c.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout: legacy client fscid: -1 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout: 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout: No filesystems configured 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T03:56:10.029 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Added host smithi050 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.030 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.agent.smithi050", "caps": []} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi050", "caps": []}]': finished 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Deploying daemon agent.smithi050 on smithi050 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.031 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3887315603' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3887315603' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:10.032 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b;count:3 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Deploying daemon mon.b on smithi050 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1981209828' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.033 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/528411821' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.a calling monitor election 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.b calling monitor election 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.034 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: monmap epoch 2 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: last_changed 2024-09-10T03:56:01.265123+0000 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: min_mon_release 19 (squid) 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: election_strategy: 1 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:10.035 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: fsmap 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mgrmap e14: y(active, since 38s) 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: overall HEALTH_OK 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: Deploying daemon mon.c on smithi038 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/704816649' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:10.036 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/843627315' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:10.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 ceph-mon[30767]: mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-10T03:56:10.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 bash[30748]: 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 2024-09-10T03:56:10.037 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:09 smithi038 systemd[1]: Started Ceph mon.c for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:56:10.691 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T03:56:10.692 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T03:56:10.956 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: mon.a calling monitor election 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: mon.b calling monitor election 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: monmap epoch 3 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: min_mon_release 19 (squid) 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: election_strategy: 1 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:15.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: fsmap 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: mgrmap e14: y(active, since 47s) 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: mon.c (rank 2) addr [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] is down (out of quorum) 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.008 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: mon.a calling monitor election 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: mon.b calling monitor election 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: monmap epoch 3 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: min_mon_release 19 (squid) 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: election_strategy: 1 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:15.056 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: fsmap 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: mgrmap e14: y(active, since 47s) 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: mon.c (rank 2) addr [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] is down (out of quorum) 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.057 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.058 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.263 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:56:15.263 INFO:teuthology.orchestra.run.smithi050.stdout:{"epoch":3,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","modified":"2024-09-10T03:56:09.661057Z","created":"2024-09-10T03:54:24.197396Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:3300","nonce":0},{"type":"v1","addr":"172.21.15.50:6789","nonce":0}]},"addr":"172.21.15.50:6789/0","public_addr":"172.21.15.50:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3301","nonce":0},{"type":"v1","addr":"172.21.15.38:6790","nonce":0}]},"addr":"172.21.15.38:6790/0","public_addr":"172.21.15.38:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T03:56:15.263 INFO:teuthology.orchestra.run.smithi050.stderr:dumped monmap epoch 3 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:15.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: Reconfiguring daemon mon.a on smithi038 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/3940271890' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:15.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: Reconfiguring daemon mon.a on smithi038 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/3940271890' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:16.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.437 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T03:56:16.437 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T03:56:16.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:16.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:56:16.922 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: mon.a calling monitor election 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: mon.b calling monitor election 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: monmap epoch 3 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:16.923 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: min_mon_release 19 (squid) 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: election_strategy: 1 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: fsmap 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: mgrmap e14: y(active, since 47s) 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: mon.c (rank 2) addr [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] is down (out of quorum) 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.924 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.925 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: Reconfiguring daemon mon.a on smithi038 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/3940271890' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-10T03:56:16.926 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:17.386 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mon.c calling monitor election 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mon.c calling monitor election 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mon.a calling monitor election 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mon.b calling monitor election 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: monmap epoch 3 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: min_mon_release 19 (squid) 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: election_strategy: 1 2024-09-10T03:56:17.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: fsmap 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: mgrmap e14: y(active, since 49s) 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T03:56:17.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:17 smithi050 ceph-mon[26687]: overall HEALTH_OK 2024-09-10T03:56:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mon.c calling monitor election 2024-09-10T03:56:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mon.c calling monitor election 2024-09-10T03:56:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mon.a calling monitor election 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mon.b calling monitor election 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: monmap epoch 3 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: min_mon_release 19 (squid) 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: election_strategy: 1 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: fsmap 2024-09-10T03:56:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: mgrmap e14: y(active, since 49s) 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[25393]: overall HEALTH_OK 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mon.c calling monitor election 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mon.c calling monitor election 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mon.a calling monitor election 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mon.b calling monitor election 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: monmap epoch 3 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: fsid 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: last_changed 2024-09-10T03:56:09.661057+0000 2024-09-10T03:56:17.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: created 2024-09-10T03:54:24.197396+0000 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: min_mon_release 19 (squid) 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: election_strategy: 1 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: 1: [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] mon.b 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: 2: [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] mon.c 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: fsmap 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: mgrmap e14: y(active, since 49s) 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T03:56:17.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:17 smithi038 ceph-mon[30767]: overall HEALTH_OK 2024-09-10T03:56:18.282 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:18.282 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-09-10T03:56:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T03:56:18.282 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.50:3300/0,v1:172.21.15.50:6789/0] [v2:172.21.15.38:3301/0,v1:172.21.15.38:6790/0] 2024-09-10T03:56:18.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:18.374 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:18.375 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:18.375 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:18.375 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.538 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.538 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.538 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:56:18.539 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.015 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T03:56:19.015 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:56:19.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T03:56:19.044 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:56:19.044 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:56:19.110 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:56:19.110 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T03:56:19.141 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:56:19.142 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T03:56:19.207 INFO:tasks.cephadm:Adding mgr.y on smithi038 2024-09-10T03:56:19.207 INFO:tasks.cephadm:Adding mgr.x on smithi050 2024-09-10T03:56:19.208 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi038=y;smithi050=x' 2024-09-10T03:56:19.496 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: Reconfiguring mon.c (monmap changed)... 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: Reconfiguring daemon mon.c on smithi038 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2474343510' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: Reconfiguring mon.c (monmap changed)... 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: Reconfiguring daemon mon.c on smithi038 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2474343510' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: Reconfiguring mon.c (monmap changed)... 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: Reconfiguring daemon mon.c on smithi038 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:19.570 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2474343510' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:19.571 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.571 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:19.571 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:20.170 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled mgr update... 2024-09-10T03:56:20.724 DEBUG:teuthology.orchestra.run.smithi050:mgr.x> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T03:56:20.726 DEBUG:tasks.cephadm:set 0 configs 2024-09-10T03:56:20.726 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T03:56:20.726 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T03:56:20.726 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:56:20.744 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T03:56:20.744 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/nvme4n1 2024-09-10T03:56:20.802 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/nvme4n1 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:56:18.705479921 +0000 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:24.702864583 +0000 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:24.702864583 +0000 2024-09-10T03:56:20.803 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:24.697864681 +0000 2024-09-10T03:56:20.803 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-10T03:56:20.870 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:56:20.870 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:56:20.870 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000312858 s, 1.6 MB/s 2024-09-10T03:56:20.871 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-10T03:56:20.929 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/nvme3n1 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/nvme3n1 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:56:18.693480155 +0000 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:24.564867276 +0000 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:24.564867276 +0000 2024-09-10T03:56:20.985 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:24.559867373 +0000 2024-09-10T03:56:20.986 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[25393]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[25393]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi038=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[25393]: Saving service mgr spec with placement smithi038=y;smithi050=x;count:2 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.049 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[30767]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:21.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[30767]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi038=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:21.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[30767]: Saving service mgr spec with placement smithi038=y;smithi050=x;count:2 2024-09-10T03:56:21.050 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:56:21.051 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:56:21.052 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.00022833 s, 2.2 MB/s 2024-09-10T03:56:21.053 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-10T03:56:21.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:20 smithi050 ceph-mon[26687]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:21.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:20 smithi050 ceph-mon[26687]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi038=y;smithi050=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:21.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:20 smithi050 ceph-mon[26687]: Saving service mgr spec with placement smithi038=y;smithi050=x;count:2 2024-09-10T03:56:21.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:21.112 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/nvme2n1 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/nvme2n1 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:56:18.682480368 +0000 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:24.421870066 +0000 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:24.421870066 +0000 2024-09-10T03:56:21.170 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:24.415870182 +0000 2024-09-10T03:56:21.171 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-10T03:56:21.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:56:21.235 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:56:21.235 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000297648 s, 1.7 MB/s 2024-09-10T03:56:21.236 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-10T03:56:21.293 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/nvme1n1 2024-09-10T03:56:21.349 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/nvme1n1 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-10 03:56:18.671480582 +0000 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-10 03:53:24.277872875 +0000 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-10 03:53:24.277872875 +0000 2024-09-10T03:56:21.350 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-10 03:53:24.272872973 +0000 2024-09-10T03:56:21.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-10T03:56:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-10T03:56:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-10T03:56:21.414 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000316997 s, 1.6 MB/s 2024-09-10T03:56:21.415 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-10T03:56:21.472 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T03:56:21.472 DEBUG:teuthology.orchestra.run.smithi050:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T03:56:21.493 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T03:56:21.493 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/nvme4n1 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/nvme4n1 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:21.551 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:56:18.214377905 +0000 2024-09-10T03:56:21.552 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:26.341502970 +0000 2024-09-10T03:56:21.552 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:26.341502970 +0000 2024-09-10T03:56:21.552 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:26.339503018 +0000 2024-09-10T03:56:21.552 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-10T03:56:21.621 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:56:21.621 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:56:21.622 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000154305 s, 3.3 MB/s 2024-09-10T03:56:21.623 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-10T03:56:21.681 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/nvme3n1 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/nvme3n1 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 1028 Links: 1 Device type: 103,6 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:56:18.206378097 +0000 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:26.198506403 +0000 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:26.198506403 +0000 2024-09-10T03:56:21.738 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:26.195506474 +0000 2024-09-10T03:56:21.738 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-10T03:56:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:56:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:56:21.803 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000150436 s, 3.4 MB/s 2024-09-10T03:56:21.804 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-10T03:56:21.861 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/nvme2n1 2024-09-10T03:56:21.917 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/nvme2n1 2024-09-10T03:56:21.917 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:56:18.197378312 +0000 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:26.054509859 +0000 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:26.054509859 +0000 2024-09-10T03:56:21.918 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:26.052509907 +0000 2024-09-10T03:56:21.918 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-10T03:56:21.981 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:56:21.982 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:56:21.982 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000127714 s, 4.0 MB/s 2024-09-10T03:56:21.983 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-10T03:56:22.040 DEBUG:teuthology.orchestra.run.smithi050:> stat /dev/nvme1n1 2024-09-10T03:56:22.096 INFO:teuthology.orchestra.run.smithi050.stdout: File: /dev/nvme1n1 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Access: 2024-09-10 03:56:18.189378504 +0000 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Modify: 2024-09-10 03:53:25.914513219 +0000 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout:Change: 2024-09-10 03:53:25.914513219 +0000 2024-09-10T03:56:22.097 INFO:teuthology.orchestra.run.smithi050.stdout: Birth: 2024-09-10 03:53:25.911513291 +0000 2024-09-10T03:56:22.097 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-10T03:56:22.161 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records in 2024-09-10T03:56:22.161 INFO:teuthology.orchestra.run.smithi050.stderr:1+0 records out 2024-09-10T03:56:22.162 INFO:teuthology.orchestra.run.smithi050.stderr:512 bytes copied, 0.000121779 s, 4.2 MB/s 2024-09-10T03:56:22.163 DEBUG:teuthology.orchestra.run.smithi050:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-10T03:56:22.221 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/nvme1n1... 2024-09-10T03:56:22.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme1n1 2024-09-10T03:56:22.472 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: Reconfiguring mon.b (monmap changed)... 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:22.835 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:22.836 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: Reconfiguring daemon mon.b on smithi050 2024-09-10T03:56:22.836 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.836 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.836 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: Reconfiguring mon.b (monmap changed)... 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: Reconfiguring daemon mon.b on smithi050 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: Reconfiguring mon.b (monmap changed)... 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: Reconfiguring daemon mon.b on smithi050 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:22.896 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.219 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.219 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.219 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.220 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:23 smithi050 ceph-mon[26687]: Deploying daemon mgr.x on smithi050 2024-09-10T03:56:24.222 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.222 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.222 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:24.222 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[30767]: Deploying daemon mgr.x on smithi050 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:24.223 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:24.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:24.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T03:56:24.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:24.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:24.224 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:23 smithi038 ceph-mon[25393]: Deploying daemon mgr.x on smithi050 2024-09-10T03:56:24.805 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:24 smithi050 podman[28538]: 2024-09-10 03:56:24.750979829 +0000 UTC m=+0.133425560 container create f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T03:56:24.832 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:56:24.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi038:/dev/nvme1n1 2024-09-10T03:56:25.118 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:56:25.241 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:24 smithi038 ceph-mon[25393]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:25.241 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:24 smithi038 ceph-mon[30767]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:25.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:24 smithi050 ceph-mon[26687]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:25.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:24 smithi050 podman[28538]: 2024-09-10 03:56:24.953157778 +0000 UTC m=+0.335603468 container init f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T03:56:25.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:24 smithi050 podman[28538]: 2024-09-10 03:56:24.95698709 +0000 UTC m=+0.339432779 container start f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T03:56:25.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:24 smithi050 ceph-mgr[28558]: -- 172.21.15.50:0/3729632012 <== mon.1 v2:172.21.15.50:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1fe95b0e0 con 0x55e1fe939400 2024-09-10T03:56:25.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:25 smithi050 bash[28538]: f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 2024-09-10T03:56:25.305 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 03:56:25 smithi050 systemd[1]: Started Ceph mgr.x for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T03:56:26.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: Reconfiguring daemon mgr.y on smithi038 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:26.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T03:56:26.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: Reconfiguring daemon mgr.y on smithi038 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr services"} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: Reconfiguring daemon mgr.y on smithi038 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:26.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:27.371 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:27.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:27.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:27.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:27.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:28.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:28 smithi050 ceph-mon[26687]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:28.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:28 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:28.439 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:28 smithi038 ceph-mon[25393]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:28.439 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:28.440 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:28 smithi038 ceph-mon[30767]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T03:56:28.440 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:28 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3173336558' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"}]': finished 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3173336558' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"}]': finished 2024-09-10T03:56:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3173336558' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"} : dispatch 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "808d5dc7-0ff8-439f-b0fa-93551f02dc05"}]': finished 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:29.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:30.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:30 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3809815931' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:56:30.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:30 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3809815931' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:56:30.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:30 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3809815931' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:56:31.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:31.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: Standby manager daemon x started 2024-09-10T03:56:31.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T03:56:31.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T03:56:31.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: Standby manager daemon x started 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T03:56:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:31 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: Standby manager daemon x started 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T03:56:31.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:31 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/2209559166' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T03:56:32.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:32 smithi038 ceph-mon[25393]: mgrmap e15: y(active, since 63s), standbys: x 2024-09-10T03:56:32.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:32 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T03:56:32.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:32 smithi038 ceph-mon[30767]: mgrmap e15: y(active, since 63s), standbys: x 2024-09-10T03:56:32.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:32 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T03:56:32.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:32 smithi050 ceph-mon[26687]: mgrmap e15: y(active, since 63s), standbys: x 2024-09-10T03:56:32.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:32 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T03:56:33.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:33 smithi038 ceph-mon[25393]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:33.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:33 smithi038 ceph-mon[30767]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:33.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:33 smithi050 ceph-mon[26687]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:35.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:35 smithi038 ceph-mon[25393]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:35.424 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:35 smithi038 ceph-mon[30767]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:35.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:35 smithi050 ceph-mon[26687]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:37.339 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[30767]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:37.339 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:37.339 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:37.339 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[25393]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:37.339 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:37.340 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:37.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:37 smithi050 ceph-mon[26687]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:38.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:38 smithi050 ceph-mon[26687]: Deploying daemon osd.0 on smithi038 2024-09-10T03:56:38.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:38 smithi038 ceph-mon[25393]: Deploying daemon osd.0 on smithi038 2024-09-10T03:56:38.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:38 smithi038 ceph-mon[30767]: Deploying daemon osd.0 on smithi038 2024-09-10T03:56:39.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:39 smithi050 ceph-mon[26687]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:39.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:39 smithi038 ceph-mon[25393]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:39.644 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:39 smithi038 ceph-mon[30767]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:40.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:40 smithi038 ceph-mon[25393]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:40.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:40 smithi038 ceph-mon[30767]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:40.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:40 smithi050 ceph-mon[26687]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: Reconfiguring osd.0 (unknown last config time)... 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:43.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[25393]: Reconfiguring daemon osd.0 on smithi038 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: Reconfiguring osd.0 (unknown last config time)... 2024-09-10T03:56:43.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:43.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:42 smithi038 ceph-mon[30767]: Reconfiguring daemon osd.0 on smithi038 2024-09-10T03:56:43.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: Reconfiguring osd.0 (unknown last config time)... 2024-09-10T03:56:43.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-10T03:56:43.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:43.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:42 smithi050 ceph-mon[26687]: Reconfiguring daemon osd.0 on smithi038 2024-09-10T03:56:44.169 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:43 smithi038 ceph-mon[25393]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:56:44.170 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:43 smithi038 ceph-mon[30767]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:56:44.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:43 smithi050 ceph-mon[26687]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:56:45.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:44 smithi038 ceph-mon[25393]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:45.160 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:44 smithi038 ceph-mon[30767]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:45.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:44 smithi050 ceph-mon[26687]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:46.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[30767]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:46 smithi038 ceph-mon[25393]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:46 smithi050 ceph-mon[26687]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:47.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:47.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:47.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:47.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:48.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:48.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:48.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:48.076 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:48.077 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:48.077 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:48.077 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[30767]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state']) 2024-09-10T03:56:49.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T03:56:49.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[25393]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state']) 2024-09-10T03:56:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T03:56:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:48 smithi050 ceph-mon[26687]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.0 on smithi038 is in unknown state']) 2024-09-10T03:56:49.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:48 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T03:56:49.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:49.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:49 smithi038 ceph-mon[25393]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:50.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:49 smithi038 ceph-mon[30767]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:50.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:49 smithi050 ceph-mon[26687]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:51.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:50 smithi038 ceph-mon[30767]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T03:56:51.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:50 smithi038 ceph-mon[25393]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T03:56:51.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:50 smithi050 ceph-mon[26687]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-10T03:56:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[25393]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[25393]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T03:56:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[25393]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T03:56:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[25393]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:56:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[30767]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[30767]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T03:56:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[30767]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T03:56:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[30767]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:56:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:52.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:51 smithi050 ceph-mon[26687]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:52.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:51 smithi050 ceph-mon[26687]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T03:56:52.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:51 smithi050 ceph-mon[26687]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T03:56:52.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:51 smithi050 ceph-mon[26687]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:56:52.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:52.528 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-09-10T03:56:53.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:56:53.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T03:56:53.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T03:56:53.151 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.152 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.182 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T03:56:53.185 INFO:tasks.cephadm:Deploying osd.1 on smithi038 with /dev/nvme2n1... 2024-09-10T03:56:53.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme2n1 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317]' entity='osd.0' 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:56:53.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:56:53.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.481 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:56:53.878 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi038 to 15044M 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317] boot 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi038 to 15044M 2024-09-10T03:56:53.879 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:53.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317] boot 2024-09-10T03:56:53.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T03:56:53.880 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:53 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:54.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi038 to 15044M 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: osd.0 [v2:172.21.15.38:6802/4169351317,v1:172.21.15.38:6803/4169351317] boot 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T03:56:54.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:53 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T03:56:56.034 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:56:56.061 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:55 smithi038 ceph-mon[25393]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:56.061 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:55 smithi038 ceph-mon[25393]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T03:56:56.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:55 smithi038 ceph-mon[30767]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:56.061 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:55 smithi038 ceph-mon[30767]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T03:56:56.062 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi038:/dev/nvme2n1 2024-09-10T03:56:56.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:55 smithi050 ceph-mon[26687]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:56.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:55 smithi050 ceph-mon[26687]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T03:56:56.316 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:56:58.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[25393]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:58.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[25393]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:58.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:58.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[30767]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[30767]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:58.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:56:58.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:57 smithi050 ceph-mon[26687]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:56:58.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:57 smithi050 ceph-mon[26687]: from='client.14253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:56:58.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:56:58.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:56:58.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:00.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[25393]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"} : dispatch 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"}]': finished 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[25393]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[30767]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"} : dispatch 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"}]': finished 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[30767]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T03:57:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:56:59 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:00.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:59 smithi050 ceph-mon[26687]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:00.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:59 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"} : dispatch 2024-09-10T03:57:00.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:59 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3759507290' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "83bc2578-eb39-48e2-8af2-c8214f4cd10d"}]': finished 2024-09-10T03:57:00.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:59 smithi050 ceph-mon[26687]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T03:57:00.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:56:59 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:01.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi038 2024-09-10T03:57:01.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/939366105' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi038 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:01 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/939366105' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:01.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:01 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi038 2024-09-10T03:57:01.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:01.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:01 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/939366105' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:02.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:02 smithi038 ceph-mon[30767]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:02.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:02 smithi038 ceph-mon[25393]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:02.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:02 smithi050 ceph-mon[26687]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:04.439 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:04 smithi038 ceph-mon[25393]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:04.439 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:04 smithi038 ceph-mon[30767]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:04.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:04 smithi050 ceph-mon[26687]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:06.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:06 smithi038 ceph-mon[25393]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:06.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:06 smithi038 ceph-mon[30767]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:06.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:06 smithi050 ceph-mon[26687]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T03:57:08.307 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:08 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.454 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:08.455 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-10T03:57:08.455 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:08 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:09.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:09 smithi050 ceph-mon[26687]: Deploying daemon osd.1 on smithi038 2024-09-10T03:57:09.556 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:09 smithi038 ceph-mon[25393]: Deploying daemon osd.1 on smithi038 2024-09-10T03:57:09.556 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:09 smithi038 ceph-mon[30767]: Deploying daemon osd.1 on smithi038 2024-09-10T03:57:10.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:10 smithi050 ceph-mon[26687]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:10.555 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:10 smithi038 ceph-mon[25393]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:10.556 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:10 smithi038 ceph-mon[30767]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:11.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:11 smithi038 ceph-mon[30767]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:11.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:11 smithi038 ceph-mon[25393]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:11.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:11 smithi050 ceph-mon[26687]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:14.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:13 smithi038 ceph-mon[30767]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:14.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:13 smithi038 ceph-mon[25393]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:14.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:13 smithi050 ceph-mon[26687]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.155 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:15 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:16.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:16.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:16.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:16.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:16.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:15 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:18.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:17 smithi038 ceph-mon[25393]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:18.145 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:17 smithi038 ceph-mon[30767]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:18.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:17 smithi050 ceph-mon[26687]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:20.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[25393]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:20.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi038 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[30767]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi038 2024-09-10T03:57:20.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:19 smithi050 ceph-mon[26687]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:19 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi038 2024-09-10T03:57:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.372 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 1 on host 'smithi038' 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.974 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[30767]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:20.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:20.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:20.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:20 smithi038 ceph-mon[25393]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T03:57:20.977 DEBUG:teuthology.orchestra.run.smithi038:osd.1> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T03:57:20.979 INFO:tasks.cephadm:Deploying osd.2 on smithi038 with /dev/nvme3n1... 2024-09-10T03:57:20.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme3n1 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:20 smithi050 ceph-mon[26687]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-10T03:57:22.174 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi038 to 7522M 2024-09-10T03:57:22.174 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:22.174 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T03:57:22.174 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi038 to 7522M 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:22.175 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:22.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi038 to 7522M 2024-09-10T03:57:22.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:22.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T03:57:22.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T03:57:22.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:22.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:22.452 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:22 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1[37994]: 2024-09-10T03:57:22.404+0000 7f88891fd640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T03:57:22.452 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:22 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1[37994]: 2024-09-10T03:57:22.407+0000 7f8885024640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:57:22.582 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:57:23.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[25393]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[25393]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[25393]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[30767]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[30767]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:23 smithi038 ceph-mon[30767]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:23.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:23 smithi050 ceph-mon[26687]: from='osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:23 smithi050 ceph-mon[26687]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T03:57:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:23 smithi050 ceph-mon[26687]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T03:57:24.658 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:57:24.658 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[25393]: osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551] boot 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[25393]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[30767]: osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551] boot 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[30767]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T03:57:24.659 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:24 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:24.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:24 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:57:24.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:24 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:57:24.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:24 smithi050 ceph-mon[26687]: osd.1 [v2:172.21.15.38:6810/1075185551,v1:172.21.15.38:6811/1075185551] boot 2024-09-10T03:57:24.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:24 smithi050 ceph-mon[26687]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T03:57:24.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:24 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T03:57:25.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:25 smithi038 ceph-mon[25393]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T03:57:25.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:25 smithi038 ceph-mon[25393]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:25.612 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:25 smithi038 ceph-mon[30767]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T03:57:25.612 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:25 smithi038 ceph-mon[30767]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:25.615 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:57:25.646 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi038:/dev/nvme3n1 2024-09-10T03:57:25.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:25 smithi050 ceph-mon[26687]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T03:57:25.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:25 smithi050 ceph-mon[26687]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:25.905 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:26.950 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:26 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:27.047 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:27.047 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:27.047 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:26 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:27.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:27 smithi038 ceph-mon[25393]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:27.897 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:27 smithi038 ceph-mon[25393]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:27.897 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:27 smithi038 ceph-mon[30767]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:27.898 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:27 smithi038 ceph-mon[30767]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:27.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:27 smithi050 ceph-mon[26687]: from='client.24173 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:27.943 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:27 smithi050 ceph-mon[26687]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:29.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"} : dispatch 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"}]': finished 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T03:57:29.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"} : dispatch 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"}]': finished 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T03:57:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"} : dispatch 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/1409725540' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cb38e881-abe9-449e-9ac6-ebe02bcfdfa9"}]': finished 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T03:57:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:30.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:30 smithi038 ceph-mon[25393]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:30.216 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:30 smithi038 ceph-mon[30767]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:30.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:30 smithi050 ceph-mon[26687]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:31.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:31 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/812722443' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:31.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:31 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/812722443' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:31.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:31 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/812722443' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:32.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:32 smithi038 ceph-mon[25393]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:32.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:32 smithi038 ceph-mon[30767]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:32.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:32 smithi050 ceph-mon[26687]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:34 smithi038 ceph-mon[25393]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:34.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:34 smithi038 ceph-mon[30767]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:34.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:34 smithi050 ceph-mon[26687]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:36.391 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:36 smithi038 ceph-mon[25393]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:36.391 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:36 smithi038 ceph-mon[30767]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:36.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:36 smithi050 ceph-mon[26687]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:37.412 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T03:57:37.412 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.413 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-10T03:57:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:37.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:38.336 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:38 smithi038 ceph-mon[25393]: Deploying daemon osd.2 on smithi038 2024-09-10T03:57:38.336 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:38 smithi038 ceph-mon[25393]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:38.337 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:38 smithi038 ceph-mon[30767]: Deploying daemon osd.2 on smithi038 2024-09-10T03:57:38.337 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:38 smithi038 ceph-mon[30767]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:38.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:38 smithi050 ceph-mon[26687]: Deploying daemon osd.2 on smithi038 2024-09-10T03:57:38.456 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:38 smithi050 ceph-mon[26687]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:40.482 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:40 smithi038 ceph-mon[25393]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:40.482 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:40 smithi038 ceph-mon[30767]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:40.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:40 smithi050 ceph-mon[26687]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:42.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:42 smithi038 ceph-mon[25393]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:42.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:42 smithi038 ceph-mon[30767]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:42.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:42 smithi050 ceph-mon[26687]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:44.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:44 smithi038 ceph-mon[25393]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:44.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:44 smithi038 ceph-mon[30767]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:44.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:44 smithi050 ceph-mon[26687]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.452 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.453 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:45 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:45 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:46.426 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:46 smithi038 ceph-mon[25393]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:46.426 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:46 smithi038 ceph-mon[30767]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:46.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:46 smithi050 ceph-mon[26687]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:48.381 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:48 smithi050 ceph-mon[26687]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:48.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:48 smithi038 ceph-mon[30767]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:48.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:48 smithi038 ceph-mon[25393]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:49.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:49.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:50.278 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 2 on host 'smithi038' 2024-09-10T03:57:50.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[25393]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:50.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[25393]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[25393]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[30767]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:50.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[30767]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:50 smithi038 ceph-mon[30767]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:50 smithi050 ceph-mon[26687]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:50.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:50 smithi050 ceph-mon[26687]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:50 smithi050 ceph-mon[26687]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-10T03:57:50.953 DEBUG:teuthology.orchestra.run.smithi038:osd.2> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T03:57:50.956 INFO:tasks.cephadm:Deploying osd.3 on smithi038 with /dev/nvme4n1... 2024-09-10T03:57:50.956 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme4n1 2024-09-10T03:57:51.255 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:57:51.497 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2[41372]: 2024-09-10T03:57:51.225+0000 7fc056c71640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T03:57:51.498 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2[41372]: 2024-09-10T03:57:51.230+0000 7fc052a98640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.498 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T03:57:51.499 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:51.500 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:51 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:51.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:51 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi038 to 5014M 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi038 to 5014M 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:52.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:52.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T03:57:52.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:52 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi038 to 5014M 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: pgmap v81: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:52.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:52 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[25393]: osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505] boot 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[25393]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:53.491 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:57:53.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:57:53.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[30767]: osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505] boot 2024-09-10T03:57:53.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[30767]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T03:57:53.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:53 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:53.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:53 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:57:53.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:53 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:57:53.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:53 smithi050 ceph-mon[26687]: osd.2 [v2:172.21.15.38:6818/1120523505,v1:172.21.15.38:6819/1120523505] boot 2024-09-10T03:57:53.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:53 smithi050 ceph-mon[26687]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T03:57:53.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:53 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T03:57:53.732 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T03:57:53.760 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi038:/dev/nvme4n1 2024-09-10T03:57:54.016 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T03:57:54.466 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[30767]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:54.466 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[30767]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T03:57:54.466 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:54.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[25393]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:54.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[25393]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T03:57:54.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:54 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:54.557 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:54 smithi050 ceph-mon[26687]: pgmap v84: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T03:57:54.557 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:54 smithi050 ceph-mon[26687]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T03:57:54.557 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:54 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:55.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:55.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T03:57:55.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[25393]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:55.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:55.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:55.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:55 smithi038 ceph-mon[30767]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:55.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:55.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='client.14307 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:55.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:55 smithi050 ceph-mon[26687]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42716]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42716]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42716]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42716]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42721]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42721]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.288 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42721]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42689]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42689]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42689]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42700]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42700]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42700]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.289 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42700]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.620 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42724]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42724]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42724]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42724]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42727]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42727]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42727]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.621 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42727]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.621 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[25393]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T03:57:56.621 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[25393]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42730]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42730]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42730]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42730]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42733]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42733]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42733]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.622 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42733]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[30767]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[30767]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42743]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42746]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42746]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42746]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.623 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:56 smithi038 sudo[42746]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 ceph-mon[26687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 ceph-mon[26687]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29900]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29900]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29900]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29900]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29903]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T03:57:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29903]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T03:57:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29903]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T03:57:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:56 smithi050 sudo[29903]: pam_unix(sudo:session): session closed for user root 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.490 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:57.491 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"} : dispatch 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"}]': finished 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi038 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T03:57:57.492 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.493 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"}]': finished 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi038 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:57.494 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.495 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:57 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: pgmap v90: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"} : dispatch 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3184410272' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "285121c6-325c-4c54-9fe3-a096a4acbf0a"}]': finished 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi038 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:57 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T03:57:58.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:58.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:57:58.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:57:59.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:59 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3742332538' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:59.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:59 smithi050 ceph-mon[26687]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-10T03:57:59.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:57:59 smithi050 ceph-mon[26687]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3742332538' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[30767]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-10T03:57:59.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[30767]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:57:59.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3742332538' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:57:59.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[25393]: mgrmap e16: y(active, since 2m), standbys: x 2024-09-10T03:57:59.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:57:59 smithi038 ceph-mon[25393]: pgmap v92: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:02.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:01 smithi038 ceph-mon[25393]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:02.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:01 smithi038 ceph-mon[30767]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:02.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:01 smithi050 ceph-mon[26687]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:03.957 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:03 smithi038 ceph-mon[25393]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:03.957 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:03 smithi038 ceph-mon[30767]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:04.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:03 smithi050 ceph-mon[26687]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:06.170 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:05 smithi038 ceph-mon[25393]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:06.171 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:05 smithi038 ceph-mon[30767]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:06.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:05 smithi050 ceph-mon[26687]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:08.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[25393]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[25393]: Deploying daemon osd.3 on smithi038 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[30767]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:08.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:07 smithi038 ceph-mon[30767]: Deploying daemon osd.3 on smithi038 2024-09-10T03:58:08.156 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:07 smithi050 ceph-mon[26687]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:08.156 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:07 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-10T03:58:08.156 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:07 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:08.156 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:07 smithi050 ceph-mon[26687]: Deploying daemon osd.3 on smithi038 2024-09-10T03:58:09.660 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.660 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.660 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.660 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[30767]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:09.660 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:09 smithi038 ceph-mon[25393]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:09 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:09 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:09 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:09 smithi050 ceph-mon[26687]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:12.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:11 smithi038 ceph-mon[25393]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:12.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:11 smithi038 ceph-mon[30767]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:12.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:11 smithi050 ceph-mon[26687]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:14.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:13 smithi038 ceph-mon[25393]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:14.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:13 smithi038 ceph-mon[30767]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:14.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:13 smithi050 ceph-mon[26687]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:15.216 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:15.216 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:15.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:16.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:16 smithi038 ceph-mon[25393]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:16.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:16 smithi038 ceph-mon[30767]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:16.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:16 smithi050 ceph-mon[26687]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:18.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:18.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:18.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:18.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.542 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 3 on host 'smithi038' 2024-09-10T03:58:18.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:18.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.122 DEBUG:teuthology.orchestra.run.smithi038:osd.3> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T03:58:19.124 INFO:tasks.cephadm:Deploying osd.4 on smithi050 with /dev/nvme1n1... 2024-09-10T03:58:19.125 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme1n1 2024-09-10T03:58:19.393 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi038 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.451 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:19 smithi050 ceph-mon[26687]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T03:58:19.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi038 2024-09-10T03:58:19.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[25393]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi038 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:19.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:19 smithi038 ceph-mon[30767]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-10T03:58:20.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:20.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T03:58:20.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T03:58:20.415 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.416 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]} : dispatch 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:20.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:21.181 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T03:58:20.976+0000 7fe756565640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T03:58:21.181 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 03:58:20 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T03:58:20.987+0000 7fe74e384640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[25393]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' 2024-09-10T03:58:21.730 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:58:21.731 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T03:58:21.731 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.731 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.731 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:21.731 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:21 smithi038 ceph-mon[30767]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' 2024-09-10T03:58:21.808 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T03:58:21.833 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:21 smithi050 ceph-mon[26687]: from='osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779]' entity='osd.3' 2024-09-10T03:58:21.834 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:/dev/nvme1n1 2024-09-10T03:58:22.088 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:58:22.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:58:22.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:58:22.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:22.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779] boot 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779] boot 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T03:58:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:22.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:58:22.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:58:22.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:22.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: osd.3 [v2:172.21.15.38:6826/2567848779,v1:172.21.15.38:6827/2567848779] boot 2024-09-10T03:58:22.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T03:58:22.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T03:58:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:23 smithi050 ceph-mon[26687]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:23.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:23 smithi050 ceph-mon[26687]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:24.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[25393]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:24.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:24.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[25393]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[30767]: from='client.24202 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:24.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:23 smithi038 ceph-mon[30767]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:25.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:24 smithi038 ceph-mon[25393]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T03:58:25.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:24 smithi038 ceph-mon[30767]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T03:58:25.050 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:24 smithi050 ceph-mon[26687]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T03:58:26.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:26.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/452532769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"}]': finished 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/452532769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"}]': finished 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T03:58:26.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:25 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:26.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:26.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/452532769' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"} : dispatch 2024-09-10T03:58:26.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "11ef947d-aea1-4d8e-a2e5-65fd221509a6"}]': finished 2024-09-10T03:58:26.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T03:58:26.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:25 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:26.995 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:26 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/3221747541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:27.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:26 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/3221747541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:27.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:26 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/3221747541' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:27.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:27 smithi050 ceph-mon[26687]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:28.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:27 smithi038 ceph-mon[25393]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:28.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:27 smithi038 ceph-mon[30767]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:29.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[25393]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:29 smithi038 ceph-mon[30767]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:29.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:29.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:29 smithi050 ceph-mon[26687]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:31.939 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:31 smithi050 ceph-mon[26687]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:32.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:31 smithi038 ceph-mon[25393]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:32.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:31 smithi038 ceph-mon[30767]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:34.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:33 smithi050 ceph-mon[26687]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:34.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:33 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T03:58:34.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:33 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[25393]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T03:58:34.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:34.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[30767]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:34.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-10T03:58:34.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:33 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:35.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:34 smithi050 ceph-mon[26687]: Deploying daemon osd.4 on smithi050 2024-09-10T03:58:35.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:34 smithi038 ceph-mon[25393]: Deploying daemon osd.4 on smithi050 2024-09-10T03:58:35.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:34 smithi038 ceph-mon[30767]: Deploying daemon osd.4 on smithi050 2024-09-10T03:58:35.954 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:35 smithi050 ceph-mon[26687]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:36.156 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:35 smithi038 ceph-mon[25393]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:36.156 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:35 smithi038 ceph-mon[30767]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:38.206 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:37 smithi050 ceph-mon[26687]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:38.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:37 smithi038 ceph-mon[25393]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:38.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:37 smithi038 ceph-mon[30767]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:39.993 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[25393]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:39.993 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:39.993 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:39.993 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:39.994 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[30767]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:39.994 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:39.994 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:39.994 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:39 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:40.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:39 smithi050 ceph-mon[26687]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:40.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:39 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:40.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:39 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:40.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:39 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:40 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.113 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:41.114 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:40 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:42.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:41 smithi038 ceph-mon[25393]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:42.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:41 smithi038 ceph-mon[30767]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:42.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:41 smithi050 ceph-mon[26687]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:44.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:44 smithi038 ceph-mon[25393]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:44.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:44 smithi038 ceph-mon[30767]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:44.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:44 smithi050 ceph-mon[26687]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:45.661 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 4 on host 'smithi050' 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.247 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:46 smithi050 ceph-mon[26687]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.249 DEBUG:teuthology.orchestra.run.smithi050:osd.4> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T03:58:46.252 INFO:tasks.cephadm:Deploying osd.5 on smithi050 with /dev/nvme2n1... 2024-09-10T03:58:46.252 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme2n1 2024-09-10T03:58:46.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:46.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:46.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[25393]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:46.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:46 smithi038 ceph-mon[30767]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-10T03:58:46.546 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi050 to 16068M 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:47.213 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-mon[26687]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi050 to 16068M 2024-09-10T03:58:47.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T03:58:47.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[25393]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi050 to 16068M 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: from='osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:47.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:47 smithi038 ceph-mon[30767]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:58:47.820 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T03:58:47.707+0000 7f5a9f1bf640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T03:58:47.820 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 03:58:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T03:58:47.712+0000 7f5a9afe6640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:58:48.111 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:48 smithi050 ceph-mon[26687]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:48.111 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:48 smithi050 ceph-mon[26687]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:58:48.111 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:48 smithi050 ceph-mon[26687]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T03:58:48.111 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:48.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[25393]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:48.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[25393]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:58:48.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[25393]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T03:58:48.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:48.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[30767]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 357 GiB / 358 GiB avail 2024-09-10T03:58:48.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[30767]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:58:48.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[30767]: osdmap e30: 5 total, 4 up, 5 in 2024-09-10T03:58:48.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:48.947 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:58:48.972 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:/dev/nvme2n1 2024-09-10T03:58:49.752 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi050 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:49.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:49.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:49.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666] boot 2024-09-10T03:58:49.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T03:58:49.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:49.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:49 smithi050 ceph-mon[26687]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:50.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi050 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666] boot 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T03:58:50.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[25393]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi050 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: osd.4 [v2:172.21.15.50:6800/2051297666,v1:172.21.15.50:6801/2051297666] boot 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T03:58:50.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:49 smithi038 ceph-mon[30767]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:51.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:51.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:51.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:50 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:58:51.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:50 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:58:51.997 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:51 smithi050 ceph-mon[26687]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:51.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:51 smithi050 ceph-mon[26687]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T03:58:51.998 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:51 smithi050 ceph-mon[26687]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[25393]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[25393]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T03:58:52.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[25393]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[30767]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:58:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[30767]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T03:58:52.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:51 smithi038 ceph-mon[30767]: pgmap v128: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:53.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:52 smithi038 ceph-mon[25393]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T03:58:53.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:52 smithi038 ceph-mon[30767]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T03:58:53.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:52 smithi050 ceph-mon[26687]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T03:58:54.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:53 smithi038 ceph-mon[25393]: osdmap e35: 5 total, 5 up, 5 in 2024-09-10T03:58:54.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:53 smithi038 ceph-mon[25393]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:54.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:53 smithi038 ceph-mon[30767]: osdmap e35: 5 total, 5 up, 5 in 2024-09-10T03:58:54.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:53 smithi038 ceph-mon[30767]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:54.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:53 smithi050 ceph-mon[26687]: osdmap e35: 5 total, 5 up, 5 in 2024-09-10T03:58:54.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:53 smithi050 ceph-mon[26687]: pgmap v131: 1 pgs: 1 remapped+peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:58:55.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1659273914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"}]': finished 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[25393]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1659273914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"}]': finished 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[30767]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T03:58:55.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:54 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:58:55.040 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:54 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1659273914' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:54 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"} : dispatch 2024-09-10T03:58:55.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:54 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c059913f-4e04-4414-b51b-f26db0e9f7dd"}]': finished 2024-09-10T03:58:55.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:54 smithi050 ceph-mon[26687]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T03:58:55.041 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:54 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:58:56.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:55 smithi038 ceph-mon[30767]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:58:56.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:55 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/3654884837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:56.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:55 smithi038 ceph-mon[25393]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:58:56.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:55 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/3654884837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:56.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:55 smithi050 ceph-mon[26687]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:58:56.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:55 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/3654884837' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:58:58.238 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:57 smithi038 ceph-mon[25393]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:58:58.239 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:57 smithi038 ceph-mon[30767]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:58:58.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:57 smithi050 ceph-mon[26687]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 136 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:59:00.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:58:59 smithi038 ceph-mon[25393]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:00.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:58:59 smithi038 ceph-mon[30767]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:00.154 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:58:59 smithi050 ceph-mon[26687]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:01.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:01.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:02.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:02 smithi050 ceph-mon[26687]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:02.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:02 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T03:59:02.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:02 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:02.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[25393]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:02.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T03:59:02.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:02.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[30767]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:02.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-10T03:59:02.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:02 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:03.435 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:03 smithi050 ceph-mon[26687]: Deploying daemon osd.5 on smithi050 2024-09-10T03:59:03.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:03 smithi038 ceph-mon[25393]: Deploying daemon osd.5 on smithi050 2024-09-10T03:59:03.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:03 smithi038 ceph-mon[30767]: Deploying daemon osd.5 on smithi050 2024-09-10T03:59:04.435 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:04 smithi050 ceph-mon[26687]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T03:59:04.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:04 smithi038 ceph-mon[25393]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T03:59:04.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:04 smithi038 ceph-mon[30767]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T03:59:06.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:06 smithi038 ceph-mon[25393]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T03:59:06.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:06 smithi038 ceph-mon[30767]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T03:59:06.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:06 smithi050 ceph-mon[26687]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-09-10T03:59:08.377 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:08 smithi050 ceph-mon[26687]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:08.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:08 smithi038 ceph-mon[25393]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:08.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:08 smithi038 ceph-mon[30767]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:10.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:12.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:12 smithi038 ceph-mon[25393]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:12.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:12 smithi038 ceph-mon[30767]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:12.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:12 smithi050 ceph-mon[26687]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:13.952 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 5 on host 'smithi050' 2024-09-10T03:59:14.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:14.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:14.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:14 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:14 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:14.656 DEBUG:teuthology.orchestra.run.smithi050:osd.5> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T03:59:14.658 INFO:tasks.cephadm:Deploying osd.6 on smithi050 with /dev/nvme3n1... 2024-09-10T03:59:14.658 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme3n1 2024-09-10T03:59:15.152 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:59:15.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:15 smithi038 ceph-mon[25393]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:15.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:15 smithi038 ceph-mon[25393]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:15.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:15 smithi038 ceph-mon[30767]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:15.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:15 smithi038 ceph-mon[30767]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:15.528 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:15 smithi050 ceph-mon[26687]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:15.528 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:15 smithi050 ceph-mon[26687]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.383 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:16.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:16.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:16.384 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:16.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:16.507 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:16 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:16.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T03:59:16.540+0000 7f808983f640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T03:59:16.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 03:59:16 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T03:59:16.545+0000 7f808165e640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:59:17.339 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:17 smithi050 ceph-mon[26687]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:17.340 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:17 smithi050 ceph-mon[26687]: osdmap e38: 6 total, 5 up, 6 in 2024-09-10T03:59:17.340 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:17 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.340 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:17 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.340 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:17 smithi050 ceph-mon[26687]: from='osd.5 ' entity='osd.5' 2024-09-10T03:59:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[25393]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[25393]: osdmap e38: 6 total, 5 up, 6 in 2024-09-10T03:59:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[25393]: from='osd.5 ' entity='osd.5' 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[30767]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[30767]: osdmap e38: 6 total, 5 up, 6 in 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:17.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:17 smithi038 ceph-mon[30767]: from='osd.5 ' entity='osd.5' 2024-09-10T03:59:18.188 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:59:18.217 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:/dev/nvme3n1 2024-09-10T03:59:18.474 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:59:18.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:59:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:59:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226] boot 2024-09-10T03:59:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T03:59:18.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:18.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:59:18.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:59:18.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226] boot 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: osd.5 [v2:172.21.15.50:6808/765649226,v1:172.21.15.50:6809/765649226] boot 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T03:59:18.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T03:59:19.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[30767]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[30767]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[25393]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:19.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:19 smithi038 ceph-mon[25393]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T03:59:19.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:19 smithi050 ceph-mon[26687]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T03:59:19.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:19.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:19.527 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:19 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:19.528 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:19 smithi050 ceph-mon[26687]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T03:59:20.472 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:20 smithi050 ceph-mon[26687]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:20.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:20 smithi050 ceph-mon[26687]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:20.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:20 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[25393]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[25393]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.572 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[30767]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:20.573 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[30767]: from='client.24256 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:20.573 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.573 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:20.573 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:20 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:21.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:21 smithi038 ceph-mon[25393]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T03:59:21.369 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:21 smithi038 ceph-mon[30767]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T03:59:21.402 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:21 smithi050 ceph-mon[26687]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: pgmap v152: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/429848687' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"}]': finished 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T03:59:22.530 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:22.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: pgmap v152: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/429848687' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"}]': finished 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: pgmap v152: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T03:59:22.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/429848687' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"} : dispatch 2024-09-10T03:59:22.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c4169b6-e4b7-48ab-bf8c-4860d1c45b25"}]': finished 2024-09-10T03:59:22.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T03:59:22.756 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:23.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:23 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1534903465' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:23.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:23 smithi038 ceph-mon[25393]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:23 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1534903465' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:23.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:23 smithi038 ceph-mon[30767]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:23.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:23 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1534903465' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:23.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:23 smithi050 ceph-mon[26687]: pgmap v155: 1 pgs: 1 remapped+peering; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:25.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:24 smithi038 ceph-mon[25393]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T03:59:25.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:24 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T03:59:25.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:24 smithi038 ceph-mon[30767]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T03:59:25.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:24 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T03:59:25.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:24 smithi050 ceph-mon[26687]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T03:59:25.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:24 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T03:59:26.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:25 smithi050 ceph-mon[26687]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T03:59:26.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:25 smithi038 ceph-mon[25393]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T03:59:26.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:25 smithi038 ceph-mon[30767]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T03:59:28.080 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:27 smithi050 ceph-mon[26687]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T03:59:28.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:27 smithi038 ceph-mon[25393]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T03:59:28.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:27 smithi038 ceph-mon[30767]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T03:59:29.923 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:29 smithi050 ceph-mon[26687]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:29.924 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T03:59:29.924 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:29.924 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:29 smithi050 ceph-mon[26687]: Deploying daemon osd.6 on smithi050 2024-09-10T03:59:30.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[25393]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:30.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T03:59:30.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:30.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[25393]: Deploying daemon osd.6 on smithi050 2024-09-10T03:59:30.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[30767]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:30.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-10T03:59:30.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:30.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:29 smithi038 ceph-mon[30767]: Deploying daemon osd.6 on smithi050 2024-09-10T03:59:31.962 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:31 smithi050 ceph-mon[26687]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T03:59:32.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:31 smithi038 ceph-mon[25393]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T03:59:32.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:31 smithi038 ceph-mon[30767]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-10T03:59:34.013 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:33 smithi050 ceph-mon[26687]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T03:59:34.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:33 smithi038 ceph-mon[25393]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T03:59:34.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:33 smithi038 ceph-mon[30767]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-09-10T03:59:36.128 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:35 smithi050 ceph-mon[26687]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T03:59:36.128 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.128 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.128 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[25393]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T03:59:36.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[30767]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-10T03:59:36.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:36.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.168 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi050 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.169 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:36 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi050 2024-09-10T03:59:37.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:37.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:37.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi050 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:37.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:36 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[25393]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:37 smithi038 ceph-mon[30767]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:59:38.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:38.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:37 smithi050 ceph-mon[26687]: Health check failed: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] (CEPHADM_FAILED_DAEMON) 2024-09-10T03:59:40.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:40 smithi038 ceph-mon[25393]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:40.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:40 smithi038 ceph-mon[30767]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:40.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:40 smithi050 ceph-mon[26687]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.478 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:41 smithi050 ceph-mon[26687]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[25393]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:41.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:41.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:41 smithi038 ceph-mon[30767]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-10T03:59:41.533 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 6 on host 'smithi050' 2024-09-10T03:59:42.086 DEBUG:teuthology.orchestra.run.smithi050:osd.6> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T03:59:42.088 INFO:tasks.cephadm:Deploying osd.7 on smithi050 with /dev/nvme4n1... 2024-09-10T03:59:42.088 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- lvm zap /dev/nvme4n1 2024-09-10T03:59:42.381 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi050 to 5356M 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T03:59:42.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi050 to 5356M 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.663 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:42 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.701 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T03:59:42.423+0000 7f63bb723640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T03:59:42.701 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T03:59:42.430+0000 7f63b754a640 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='osd.6 ' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi050 to 5356M 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:42.702 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:42.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T03:59:42.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:42.703 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:42 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:43.701 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:43 smithi050 ceph-mon[26687]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:43.701 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:43 smithi050 ceph-mon[26687]: osdmap e46: 7 total, 6 up, 7 in 2024-09-10T03:59:43.701 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:43 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:43.701 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:43 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:43.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[25393]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[25393]: osdmap e46: 7 total, 6 up, 7 in 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[30767]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[30767]: osdmap e46: 7 total, 6 up, 7 in 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:43.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:43 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619] boot 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:44 smithi050 ceph-mon[26687]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:44.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T03:59:44.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T03:59:44.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619] boot 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[25393]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: osd.6 [v2:172.21.15.50:6816/4014295619,v1:172.21.15.50:6817/4014295619] boot 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T03:59:44.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:44 smithi038 ceph-mon[30767]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 166 MiB used, 536 GiB / 536 GiB avail 2024-09-10T03:59:44.890 INFO:teuthology.orchestra.run.smithi050.stdout: 2024-09-10T03:59:44.918 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi050:/dev/nvme4n1 2024-09-10T03:59:45.173 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T03:59:45.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:45 smithi050 ceph-mon[26687]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T03:59:45.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:45 smithi038 ceph-mon[30767]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T03:59:45.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:45 smithi038 ceph-mon[25393]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:46 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:47.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T03:59:47.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:47.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:47.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: pgmap v177: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi050:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-10T03:59:47.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:46 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:47.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:47 smithi050 ceph-mon[26687]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T03:59:48.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:47 smithi038 ceph-mon[25393]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T03:59:48.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:47 smithi038 ceph-mon[30767]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T03:59:48.980 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: osdmap e51: 7 total, 7 up, 7 in 2024-09-10T03:59:48.980 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:48.980 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1486883503' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:48.980 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:48.981 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"}]': finished 2024-09-10T03:59:48.981 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T03:59:48.981 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T03:59:49.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: osdmap e51: 7 total, 7 up, 7 in 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1486883503' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"}]': finished 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: osdmap e51: 7 total, 7 up, 7 in 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: pgmap v180: 1 pgs: 1 remapped+peering; 577 KiB data, 193 MiB used, 626 GiB / 626 GiB avail 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1486883503' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:49.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"} : dispatch 2024-09-10T03:59:49.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0d3e6249-a211-42f4-95ee-03b1e56fc521"}]': finished 2024-09-10T03:59:49.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T03:59:49.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T03:59:49.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:49 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/242852683' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:50.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:49 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/242852683' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:50.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:49 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/242852683' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-10T03:59:51.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:50 smithi038 ceph-mon[25393]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:59:51.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:50 smithi038 ceph-mon[30767]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:59:51.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:50 smithi050 ceph-mon[26687]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 142 KiB/s, 0 objects/s recovering 2024-09-10T03:59:52.803 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:52 smithi050 ceph-mon[26687]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:59:53.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:52 smithi038 ceph-mon[25393]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:59:53.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:52 smithi038 ceph-mon[30767]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T03:59:54.695 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:54 smithi050 ceph-mon[26687]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:55.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:54 smithi038 ceph-mon[25393]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:55.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:54 smithi038 ceph-mon[30767]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-09-10T03:59:56.708 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:56 smithi050 ceph-mon[26687]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:56.709 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:56 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T03:59:56.709 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:56 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:57.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[25393]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:57.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T03:59:57.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:57.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[30767]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T03:59:57.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-10T03:59:57.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:56 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T03:59:58.368 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: Deploying daemon osd.7 on smithi050 2024-09-10T03:59:58.368 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.369 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.369 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.369 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.369 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi050 2024-09-10T03:59:58.369 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 03:59:58 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: Deploying daemon osd.7 on smithi050 2024-09-10T03:59:58.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi050 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: Deploying daemon osd.7 on smithi050 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-10T03:59:58.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi050 2024-09-10T03:59:58.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 03:59:58 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:00.267 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:00.268 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.268 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.268 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.268 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:00 smithi050 ceph-mon[26687]: daemon osd.6 on smithi050 is in unknown state 2024-09-10T04:00:00.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[25393]: daemon osd.6 on smithi050 is in unknown state 2024-09-10T04:00:00.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state'] 2024-09-10T04:00:00.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:00 smithi038 ceph-mon[30767]: daemon osd.6 on smithi050 is in unknown state 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: Detected new or changed devices on smithi050 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state']) 2024-09-10T04:00:01.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:01 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T04:00:01.380 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: Detected new or changed devices on smithi050 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state']) 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: Detected new or changed devices on smithi050 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.381 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.382 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:01.382 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s) ['daemon osd.6 on smithi050 is in unknown state']) 2024-09-10T04:00:01.382 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:01 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T04:00:03.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:03 smithi038 ceph-mon[25393]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:03.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:03 smithi038 ceph-mon[30767]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:03.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:03 smithi050 ceph-mon[26687]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:05.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:05 smithi038 ceph-mon[30767]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:05.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:05 smithi038 ceph-mon[25393]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:05.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:05 smithi050 ceph-mon[26687]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:06.481 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:06.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:06.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:06.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.482 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:06 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:06.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:06.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:06.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:06 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:07.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:07 smithi038 ceph-mon[30767]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:07.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:07 smithi038 ceph-mon[25393]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:07.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:07 smithi050 ceph-mon[26687]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:09.308 INFO:teuthology.orchestra.run.smithi050.stdout:Created osd(s) 7 on host 'smithi050' 2024-09-10T04:00:09.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:09 smithi050 ceph-mon[26687]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:09.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:09 smithi038 ceph-mon[30767]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:09.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:09 smithi038 ceph-mon[25393]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:09.958 DEBUG:teuthology.orchestra.run.smithi050:osd.7> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T04:00:09.960 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T04:00:09.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T04:00:10.232 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.510 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:10.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:10.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.511 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[25393]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.513 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:10.513 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.514 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:10 smithi038 ceph-mon[30767]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:10.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:10 smithi050 ceph-mon[26687]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-10T04:00:10.908 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: Adjusting osd_memory_target on smithi050 to 4017M 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.524 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2799215930' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:11.525 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:00:11.328+0000 7f26c9653640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T04:00:11.525 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:00:11 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:00:11.334+0000 7f26c547a640 -1 osd.7 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: Adjusting osd_memory_target on smithi050 to 4017M 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T04:00:11.592 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2799215930' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: Adjusting osd_memory_target on smithi050 to 4017M 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: from='osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]} : dispatch 2024-09-10T04:00:11.593 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:11 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2799215930' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:11.622 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":53,"num_osds":8,"num_up_osds":7,"osd_up_since":1725940783,"num_in_osds":8,"osd_in_since":1725940788,"num_remapped_pgs":0} 2024-09-10T04:00:12.624 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T04:00:12.654 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[25393]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[25393]: osdmap e54: 8 total, 7 up, 8 in 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[25393]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[30767]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[30767]: osdmap e54: 8 total, 7 up, 8 in 2024-09-10T04:00:12.655 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.656 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.656 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:12 smithi038 ceph-mon[30767]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:12 smithi050 ceph-mon[26687]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi050", "root=default"]}]': finished 2024-09-10T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:12 smithi050 ceph-mon[26687]: osdmap e54: 8 total, 7 up, 8 in 2024-09-10T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:12 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:12 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:12.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:12 smithi050 ceph-mon[26687]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 194 MiB used, 626 GiB / 626 GiB avail 2024-09-10T04:00:12.894 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:13.549 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:13.549 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: purged_snaps scrub starts 2024-09-10T04:00:13.549 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: purged_snaps scrub ok 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131] boot 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: purged_snaps scrub starts 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: purged_snaps scrub ok 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131] boot 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T04:00:13.550 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:13 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: purged_snaps scrub starts 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: purged_snaps scrub ok 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: osd.7 [v2:172.21.15.50:6824/4209821131,v1:172.21.15.50:6825/4209821131] boot 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T04:00:13.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:13 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:14.303 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":56,"num_osds":8,"num_up_osds":8,"osd_up_since":1725940812,"num_in_osds":8,"osd_in_since":1725940788,"num_remapped_pgs":1} 2024-09-10T04:00:14.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:00:14.546 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[30767]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T04:00:14.546 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/366237507' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:14.546 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[30767]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:14.546 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[30767]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T04:00:14.546 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[25393]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T04:00:14.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/366237507' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:14.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[25393]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:14.547 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:14 smithi038 ceph-mon[25393]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T04:00:14.601 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:14.676 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:14 smithi050 ceph-mon[26687]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T04:00:14.676 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:14 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/366237507' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-10T04:00:14.676 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:14 smithi050 ceph-mon[26687]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:14.676 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:14 smithi050 ceph-mon[26687]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T04:00:15.259 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:15.259 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":57,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:54:27.357206+0000","modified":"2024-09-10T04:00:14.333869+0000","last_up_change":"2024-09-10T04:00:12.324560+0000","last_in_change":"2024-09-10T03:59:48.127063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T03:57:53.950647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"808d5dc7-0ff8-439f-b0fa-93551f02dc05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6803","nonce":4169351317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6805","nonce":4169351317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6809","nonce":4169351317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6807","nonce":4169351317}]},"public_addr":"172.21.15.38:6803/4169351317","cluster_addr":"172.21.15.38:6805/4169351317","heartbeat_back_addr":"172.21.15.38:6809/4169351317","heartbeat_front_addr":"172.21.15.38:6807/4169351317","state":["exists","up"]},{"osd":1,"uuid":"83bc2578-eb39-48e2-8af2-c8214f4cd10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6811","nonce":1075185551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6813","nonce":1075185551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6817","nonce":1075185551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6815","nonce":1075185551}]},"public_addr":"172.21.15.38:6811/1075185551","cluster_addr":"172.21.15.38:6813/1075185551","heartbeat_back_addr":"172.21.15.38:6817/1075185551","heartbeat_front_addr":"172.21.15.38:6815/1075185551","state":["exists","up"]},{"osd":2,"uuid":"cb38e881-abe9-449e-9ac6-ebe02bcfdfa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6819","nonce":1120523505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6821","nonce":1120523505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6825","nonce":1120523505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6823","nonce":1120523505}]},"public_addr":"172.21.15.38:6819/1120523505","cluster_addr":"172.21.15.38:6821/1120523505","heartbeat_back_addr":"172.21.15.38:6825/1120523505","heartbeat_front_addr":"172.21.15.38:6823/1120523505","state":["exists","up"]},{"osd":3,"uuid":"285121c6-325c-4c54-9fe3-a096a4acbf0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6827","nonce":2567848779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6829","nonce":2567848779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6833","nonce":2567848779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6831","nonce":2567848779}]},"public_addr":"172.21.15.38:6827/2567848779","cluster_addr":"172.21.15.38:6829/2567848779","heartbeat_back_addr":"172.21.15.38:6833/2567848779","heartbeat_front_addr":"172.21.15.38:6831/2567848779","state":["exists","up"]},{"osd":4,"uuid":"11ef947d-aea1-4d8e-a2e5-65fd221509a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051297666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051297666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051297666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051297666}]},"public_addr":"172.21.15.50:6801/2051297666","cluster_addr":"172.21.15.50:6803/2051297666","heartbeat_back_addr":"172.21.15.50:6807/2051297666","heartbeat_front_addr":"172.21.15.50:6805/2051297666","state":["exists","up"]},{"osd":5,"uuid":"c059913f-4e04-4414-b51b-f26db0e9f7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6809","nonce":765649226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6811","nonce":765649226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6815","nonce":765649226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6813","nonce":765649226}]},"public_addr":"172.21.15.50:6809/765649226","cluster_addr":"172.21.15.50:6811/765649226","heartbeat_back_addr":"172.21.15.50:6815/765649226","heartbeat_front_addr":"172.21.15.50:6813/765649226","state":["exists","up"]},{"osd":6,"uuid":"2c4169b6-e4b7-48ab-bf8c-4860d1c45b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6817","nonce":4014295619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6819","nonce":4014295619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6823","nonce":4014295619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6821","nonce":4014295619}]},"public_addr":"172.21.15.50:6817/4014295619","cluster_addr":"172.21.15.50:6819/4014295619","heartbeat_back_addr":"172.21.15.50:6823/4014295619","heartbeat_front_addr":"172.21.15.50:6821/4014295619","state":["exists","up"]},{"osd":7,"uuid":"0d3e6249-a211-42f4-95ee-03b1e56fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6825","nonce":4209821131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6827","nonce":4209821131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6831","nonce":4209821131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6829","nonce":4209821131}]},"public_addr":"172.21.15.50:6825/4209821131","cluster_addr":"172.21.15.50:6827/4209821131","heartbeat_back_addr":"172.21.15.50:6831/4209821131","heartbeat_front_addr":"172.21.15.50:6829/4209821131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:56:51.192855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:21.374997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:50.673882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:19.689087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:46.704082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:15.345782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:42.313545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:10.773193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,6]}],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6800/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/1200911576":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/3289402189":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/2044272138":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6801/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:6800/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1803280845":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/2740767068":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1465824294":"2024-09-11T03:55:00.513768+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:00:15.582 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:15 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/4072242436' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:15.582 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:15 smithi038 ceph-mon[30767]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T04:00:15.582 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:15 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/4072242436' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:15.582 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:15 smithi038 ceph-mon[25393]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T04:00:15.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:15 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/4072242436' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:15.628 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:15 smithi050 ceph-mon[26687]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T04:00:16.016 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T03:57:53.950647+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T04:00:16.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T04:00:16.275 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:16.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:16 smithi050 ceph-mon[26687]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:16.636 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:16 smithi038 ceph-mon[25393]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:16.636 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:16 smithi038 ceph-mon[30767]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:16.917 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-10T04:00:17.674 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:17 smithi038 ceph-mon[25393]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T04:00:17.674 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:17 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3054366996' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T04:00:17.674 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:17 smithi038 ceph-mon[30767]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T04:00:17.675 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:17 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3054366996' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T04:00:17.712 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi038 2024-09-10T04:00:17.712 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi038=foo.a' 2024-09-10T04:00:17.738 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:17 smithi050 ceph-mon[26687]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T04:00:17.799 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:17 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3054366996' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-10T04:00:17.996 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:18.675 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled rgw.foo.a update... 2024-09-10T04:00:19.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:19.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.257 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:18 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: pgmap v205: 1 pgs: 1 remapped+peering; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:19.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.280 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]} : dispatch 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:19.281 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:18 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:19.309 DEBUG:teuthology.orchestra.run.smithi038:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T04:00:19.312 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi050 2024-09-10T04:00:19.312 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T04:00:19.603 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:20.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[25393]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi038=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:20.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[25393]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:20.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[25393]: Deploying daemon rgw.foo.a on smithi038 2024-09-10T04:00:20.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[30767]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi038=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:20.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[30767]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:20.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:19 smithi038 ceph-mon[30767]: Deploying daemon rgw.foo.a on smithi038 2024-09-10T04:00:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:19 smithi050 ceph-mon[26687]: from='client.24328 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi038=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:19 smithi050 ceph-mon[26687]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:20.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:19 smithi050 ceph-mon[26687]: Deploying daemon rgw.foo.a on smithi038 2024-09-10T04:00:21.000 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:00:20 smithi038 podman[48412]: 2024-09-10 04:00:20.891409501 +0000 UTC m=+0.123151242 container create 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:00:21.023 INFO:teuthology.orchestra.run.smithi050.stderr:pool 'datapool' created 2024-09-10T04:00:21.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1641378149' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1641378149' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:21 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1641378149' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} : dispatch 2024-09-10T04:00:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:21 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:21.674 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:00:21 smithi038 podman[48412]: 2024-09-10 04:00:21.346782594 +0000 UTC m=+0.578524345 container init 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:00:21.674 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:00:21 smithi038 podman[48412]: 2024-09-10 04:00:21.349971203 +0000 UTC m=+0.581712945 container start 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T04:00:21.674 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:00:21 smithi038 bash[48412]: 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 2024-09-10T04:00:21.674 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:00:21 smithi038 systemd[1]: Started Ceph rgw.foo.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:00:21.759 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T04:00:22.014 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:22.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T04:00:22.291 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:22.292 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:22 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:22.328 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T04:00:22.328 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T04:00:22.328 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:22.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:22.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:22 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:23.369 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: Checking dashboard <-> RGW credentials 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: pgmap v208: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/4190201489' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1801514863' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:23.370 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: Checking dashboard <-> RGW credentials 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: pgmap v208: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/4190201489' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1801514863' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T04:00:23.371 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:23 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: Saving service rgw.foo.a spec with placement smithi038=foo.a;count:1 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: Checking dashboard <-> RGW credentials 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: pgmap v208: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 91 KiB/s, 0 objects/s recovering 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/4190201489' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"} : dispatch 2024-09-10T04:00:23.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1801514863' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd={"prefix": "osd pool application enable","pool": "datapool","app": "rbd"} : dispatch 2024-09-10T04:00:23.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T04:00:23.556 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:23 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:24.418 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.419 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.419 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T04:00:24.419 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T04:00:24.419 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T04:00:24.419 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.420 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.420 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.420 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:24 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2647728180' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:24.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:24 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"} : dispatch 2024-09-10T04:00:25.447 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: pgmap v211: 36 pgs: 15 active+clean, 6 creating+peering, 15 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:25.448 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T04:00:25.448 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: Cluster is now healthy 2024-09-10T04:00:25.448 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.448 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.448 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[30767]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: pgmap v211: 36 pgs: 15 active+clean, 6 creating+peering, 15 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: Cluster is now healthy 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:25 smithi038 ceph-mon[25393]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: pgmap v211: 36 pgs: 15 active+clean, 6 creating+peering, 15 unknown; 577 KiB data, 221 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s; 96 KiB/s, 0 objects/s recovering 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: Cluster is now healthy 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T04:00:25.495 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:25 smithi050 ceph-mon[26687]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T04:00:25.794 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.50 --placement '1;smithi050=iscsi.a' 2024-09-10T04:00:26.050 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:26.729 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled iscsi.datapool update... 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: pgmap v214: 68 pgs: 30 active+clean, 14 creating+peering, 24 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: pgmap v214: 68 pgs: 30 active+clean, 14 creating+peering, 24 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.256 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:27 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: pgmap v214: 68 pgs: 30 active+clean, 14 creating+peering, 24 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"} : dispatch 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.306 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:27 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:27.349 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T04:00:27.349 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T04:00:27.349 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T04:00:27.380 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T04:00:27.380 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T04:00:27.407 DEBUG:teuthology.orchestra.run.smithi050:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T04:00:27.450 INFO:tasks.cephadm:Adding prometheus.a on smithi050 2024-09-10T04:00:27.450 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi050=a' 2024-09-10T04:00:27.740 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T04:00:28.123 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:28 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:28.426 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled prometheus update... 2024-09-10T04:00:28.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: from='client.24376 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.50", "placement": "1;smithi050=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: Saving service iscsi.datapool spec with placement smithi050=iscsi.a;count:1 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T04:00:28.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T04:00:28.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:28 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:29.104 DEBUG:teuthology.orchestra.run.smithi050:prometheus.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T04:00:29.107 INFO:tasks.cephadm:Adding node-exporter.a on smithi038 2024-09-10T04:00:29.107 INFO:tasks.cephadm:Adding node-exporter.b on smithi050 2024-09-10T04:00:29.107 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi038=a;smithi050=b' 2024-09-10T04:00:29.171 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: pgmap v217: 100 pgs: 47 active+clean, 8 creating+peering, 45 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:29.171 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T04:00:29.172 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.172 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.172 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.172 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:29 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:29.404 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: pgmap v217: 100 pgs: 47 active+clean, 8 creating+peering, 45 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: pgmap v217: 100 pgs: 47 active+clean, 8 creating+peering, 45 unknown; 578 KiB data, 222 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:29.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T04:00:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"} : dispatch 2024-09-10T04:00:29.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:29 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:30.110 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled node-exporter update... 2024-09-10T04:00:30.443 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: Saving service prometheus spec with placement smithi050=a;count:1 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.444 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:30 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:30.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: Saving service prometheus spec with placement smithi050=a;count:1 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: Saving service prometheus spec with placement smithi050=a;count:1 2024-09-10T04:00:30.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3770351377' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd={"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"} : dispatch 2024-09-10T04:00:30.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:30 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:30.849 DEBUG:teuthology.orchestra.run.smithi038:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T04:00:30.851 DEBUG:teuthology.orchestra.run.smithi050:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T04:00:30.892 INFO:tasks.cephadm:Adding alertmanager.a on smithi038 2024-09-10T04:00:30.892 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi038=a' 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: pgmap v220: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi038=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: Saving service node-exporter spec with placement smithi038=a;smithi050=b;count:2 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-10T04:00:31.473 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T04:00:31.474 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:31 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:31.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: pgmap v220: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi038=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: Saving service node-exporter spec with placement smithi038=a;smithi050=b;count:2 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T04:00:31.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: pgmap v220: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi038=a;smithi050=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: Saving service node-exporter spec with placement smithi038=a;smithi050=b;count:2 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2642556155' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]} : dispatch 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T04:00:31.506 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:31 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:31.757 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:32.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:32 smithi038 ceph-mon[25393]: Deploying daemon iscsi.iscsi.a on smithi050 2024-09-10T04:00:32.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:32 smithi038 ceph-mon[30767]: Deploying daemon iscsi.iscsi.a on smithi050 2024-09-10T04:00:32.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:32 smithi050 ceph-mon[26687]: Deploying daemon iscsi.iscsi.a on smithi050 2024-09-10T04:00:32.587 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled alertmanager update... 2024-09-10T04:00:33.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:33 smithi050 ceph-mon[26687]: pgmap v223: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:33.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:33 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:33.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:33 smithi038 ceph-mon[25393]: pgmap v223: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:33.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:33 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:33.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:33 smithi038 ceph-mon[30767]: pgmap v223: 132 pgs: 103 active+clean, 22 creating+peering, 7 unknown; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T04:00:33.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:33 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:33.561 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:33 smithi050 systemd[1]: Starting Ceph iscsi.iscsi.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:00:33.594 DEBUG:teuthology.orchestra.run.smithi038:alertmanager.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T04:00:33.597 INFO:tasks.cephadm:Adding grafana.a on smithi050 2024-09-10T04:00:33.597 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi050=a' 2024-09-10T04:00:33.883 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:33 smithi050 podman[45206]: 2024-09-10 04:00:33.752742653 +0000 UTC m=+0.112555666 container create 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:00:33.883 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:33 smithi050 podman[45206]: 2024-09-10 04:00:33.657128696 +0000 UTC m=+0.016941718 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:00:34.181 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:34.285 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-mon[26687]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi038=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:34.285 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-mon[26687]: Saving service alertmanager spec with placement smithi038=a;count:1 2024-09-10T04:00:34.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:33 smithi050 podman[45206]: 2024-09-10 04:00:33.996894811 +0000 UTC m=+0.356707833 container init 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T04:00:34.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 podman[45206]: 2024-09-10 04:00:34.000542031 +0000 UTC m=+0.360355043 container start 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T04:00:34.285 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 bash[45206]: 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 2024-09-10T04:00:34.286 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 systemd[1]: Started Ceph iscsi.iscsi.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:00:34.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:34 smithi038 ceph-mon[25393]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi038=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:34.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:34 smithi038 ceph-mon[25393]: Saving service alertmanager spec with placement smithi038=a;count:1 2024-09-10T04:00:34.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:34 smithi038 ceph-mon[30767]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi038=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:34.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:34 smithi038 ceph-mon[30767]: Saving service alertmanager spec with placement smithi038=a;count:1 2024-09-10T04:00:34.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Started the configuration object watcher 2024-09-10T04:00:34.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Checking for config object changes every 1s 2024-09-10T04:00:34.541 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Processing osd blocklist entries for this node 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Reading the configuration object to update local LIO configuration 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Configuration does not have an entry for this host(smithi050.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: * Environment: production 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: Use a production WSGI server instead. 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: * Debug mode: off 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug * Running on all addresses. 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T04:00:34.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: * Running on all addresses. 2024-09-10T04:00:34.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T04:00:34.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug * Running on http://[fe80::ec4:7aff:febd:1584]:5000/ (Press CTRL+C to quit) 2024-09-10T04:00:34.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: * Running on http://[fe80::ec4:7aff:febd:1584]:5000/ (Press CTRL+C to quit) 2024-09-10T04:00:35.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: pgmap v224: 132 pgs: 122 active+clean, 10 creating+peering; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 2.5 KiB/s wr, 100 op/s 2024-09-10T04:00:35.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/1480383468' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: pgmap v224: 132 pgs: 122 active+clean, 10 creating+peering; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 2.5 KiB/s wr, 100 op/s 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:35 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/1480383468' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:00:35.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: pgmap v224: 132 pgs: 122 active+clean, 10 creating+peering; 580 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 2.5 KiB/s wr, 100 op/s 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:35.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:35 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/1480383468' entity='client.iscsi.iscsi.a' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-10T04:00:36.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:36 smithi038 ceph-mon[25393]: Deploying daemon prometheus.a on smithi050 2024-09-10T04:00:36.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:36 smithi038 ceph-mon[30767]: Deploying daemon prometheus.a on smithi050 2024-09-10T04:00:36.522 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:36 smithi050 ceph-mon[26687]: Deploying daemon prometheus.a on smithi050 2024-09-10T04:00:36.536 INFO:teuthology.orchestra.run.smithi050.stdout:Scheduled grafana update... 2024-09-10T04:00:37.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:37 smithi050 ceph-mon[26687]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 244 op/s 2024-09-10T04:00:37.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:37 smithi050 ceph-mon[26687]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T04:00:37.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:37 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:37.454 DEBUG:teuthology.orchestra.run.smithi050:grafana.a> sudo journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T04:00:37.457 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T04:00:37.457 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T04:00:37.481 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[25393]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 244 op/s 2024-09-10T04:00:37.481 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[25393]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T04:00:37.482 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:37.482 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[30767]: pgmap v225: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 110 KiB/s rd, 5.5 KiB/s wr, 244 op/s 2024-09-10T04:00:37.482 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[30767]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T04:00:37.482 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:37 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:37.729 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:38.455 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-10T04:00:38.456 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBmxN9mrnj5GhAAoF+5iIJiXchhsJTRAeKbMA== 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[25393]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[25393]: Saving service grafana spec with placement smithi050=a;count:1 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[30767]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[30767]: Saving service grafana spec with placement smithi050=a;count:1 2024-09-10T04:00:38.456 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:38.457 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:38 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:38.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:38 smithi050 ceph-mon[26687]: from='client.24439 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi050=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:00:38.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:38 smithi050 ceph-mon[26687]: Saving service grafana spec with placement smithi050=a;count:1 2024-09-10T04:00:38.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:38 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:38.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:38 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:39.186 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T04:00:39.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T04:00:39.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T04:00:39.268 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T04:00:39.504 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:39 smithi050 ceph-mon[26687]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.2 KiB/s wr, 199 op/s 2024-09-10T04:00:39.504 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:39 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:39.504 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:39 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:39.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[25393]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.2 KiB/s wr, 199 op/s 2024-09-10T04:00:39.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:39.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:39.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[30767]: pgmap v226: 132 pgs: 132 active+clean; 582 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 89 KiB/s rd, 4.2 KiB/s wr, 199 op/s 2024-09-10T04:00:39.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:39.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:39 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/655386568' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:40.036 INFO:teuthology.orchestra.run.smithi050.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T04:00:41.298 INFO:teuthology.orchestra.run.smithi050.stdout:[client.1] 2024-09-10T04:00:41.299 INFO:teuthology.orchestra.run.smithi050.stdout: key = AQBpxN9mKTZeERAAtXXZNsvV+cowgqMw/qQSpg== 2024-09-10T04:00:41.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:41 smithi050 ceph-mon[26687]: pgmap v227: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-09-10T04:00:41.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.299 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:41 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[30767]: pgmap v227: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-09-10T04:00:41.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[25393]: pgmap v227: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 75 KiB/s rd, 3.7 KiB/s wr, 168 op/s 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:41.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:41 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:42.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[25393]: from='client.? 172.21.15.50:0/672268460' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.504 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[25393]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:42.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[30767]: from='client.? 172.21.15.50:0/672268460' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:42 smithi038 ceph-mon[30767]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:42.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:42 smithi050 ceph-mon[26687]: from='client.? 172.21.15.50:0/672268460' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:42 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-10T04:00:42.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:42 smithi050 ceph-mon[26687]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T04:00:42.654 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T04:00:42.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T04:00:42.654 DEBUG:teuthology.orchestra.run.smithi050:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T04:00:42.695 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph config log 1 --format=json 2024-09-10T04:00:42.962 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:43.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:43 smithi038 ceph-mon[30767]: pgmap v228: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 KiB/s wr, 142 op/s 2024-09-10T04:00:43.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:43 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:43.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:43 smithi038 ceph-mon[25393]: pgmap v228: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 KiB/s wr, 142 op/s 2024-09-10T04:00:43.505 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:43 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:43.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:43 smithi050 ceph-mon[26687]: pgmap v228: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 3.1 KiB/s wr, 142 op/s 2024-09-10T04:00:43.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:43 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:43.812 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:44.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:44 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/857831516' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T04:00:44.465 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:44 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/857831516' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T04:00:44.497 INFO:teuthology.orchestra.run.smithi038.stdout:[{"version":22,"timestamp":"2024-09-10T04:00:30.407202+0000","name":"","changes":[{"name":"mgr/mgr/dashboard/RGW_API_SECRET_KEY","new_value":"zuD8NHNVIBxFwwl3Oj3xsuf6A5YkI2uf3hhSbw5W"}]}] 2024-09-10T04:00:44.498 INFO:tasks.ceph_manager:config epoch is 22 2024-09-10T04:00:44.498 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T04:00:44.498 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T04:00:44.498 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T04:00:44.554 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:00:44.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:44 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/857831516' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-10T04:00:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:45 smithi050 ceph-mon[26687]: pgmap v229: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 2.9 KiB/s wr, 134 op/s 2024-09-10T04:00:45.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:45 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:45.688 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:45 smithi038 ceph-mon[25393]: pgmap v229: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 2.9 KiB/s wr, 134 op/s 2024-09-10T04:00:45.689 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:45 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:45.689 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:45 smithi038 ceph-mon[30767]: pgmap v229: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 60 KiB/s rd, 2.9 KiB/s wr, 134 op/s 2024-09-10T04:00:45.689 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:45 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:45.780 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:46.528 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:46 smithi050 ceph-mon[26687]: pgmap v230: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 2.1 KiB/s wr, 89 op/s 2024-09-10T04:00:46.630 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:46 smithi038 ceph-mon[30767]: pgmap v230: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 2.1 KiB/s wr, 89 op/s 2024-09-10T04:00:46.630 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:46 smithi038 ceph-mon[25393]: pgmap v230: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 40 KiB/s rd, 2.1 KiB/s wr, 89 op/s 2024-09-10T04:00:46.861 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:47.496 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:47 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/355380685' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T04:00:47.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.497 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:47 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/355380685' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T04:00:47.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.661 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/355380685' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-10T04:00:47.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.662 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:47 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:47.895 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:00:47 smithi050 systemd[1]: Started Ceph prometheus.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:00:48.162 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":3174327859},{"type":"v1","addr":"172.21.15.38:6801","nonce":3174327859}]},"active_addr":"172.21.15.38:6801/3174327859","active_change":"2024-09-10T03:55:27.477006+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":4157435586}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2980227314}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2992637904}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":259172593}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":4063531827}]}]} 2024-09-10T04:00:48.170 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T04:00:48.171 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T04:00:48.171 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:00:48.448 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:48.826 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: pgmap v231: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T04:00:48.826 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.826 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: pgmap v231: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-10T04:00:48.827 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.856 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: pgmap v231: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd={"prefix": "mgr module enable", "module": "prometheus"} : dispatch 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: ignoring --setuser ceph since I am not root 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: ignoring --setgroup ceph since I am not root 2024-09-10T04:00:48.857 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:48 smithi050 ceph-mgr[28558]: -- 172.21.15.50:0/336942340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555a242c70e0 con 0x555a242a5400 2024-09-10T04:00:49.141 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:49.141 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ignoring --setuser ceph since I am not root 2024-09-10T04:00:49.141 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:48 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ignoring --setgroup ceph since I am not root 2024-09-10T04:00:49.142 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:54:27.357206+0000","modified":"2024-09-10T04:00:30.377199+0000","last_up_change":"2024-09-10T04:00:12.324560+0000","last_in_change":"2024-09-10T03:59:48.127063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T03:57:53.950647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T04:00:20.228675+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T04:00:21.391089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T04:00:23.155552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T04:00:25.086993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T04:00:27.158517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"808d5dc7-0ff8-439f-b0fa-93551f02dc05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6803","nonce":4169351317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6805","nonce":4169351317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6809","nonce":4169351317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6807","nonce":4169351317}]},"public_addr":"172.21.15.38:6803/4169351317","cluster_addr":"172.21.15.38:6805/4169351317","heartbeat_back_addr":"172.21.15.38:6809/4169351317","heartbeat_front_addr":"172.21.15.38:6807/4169351317","state":["exists","up"]},{"osd":1,"uuid":"83bc2578-eb39-48e2-8af2-c8214f4cd10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6811","nonce":1075185551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6813","nonce":1075185551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6817","nonce":1075185551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6815","nonce":1075185551}]},"public_addr":"172.21.15.38:6811/1075185551","cluster_addr":"172.21.15.38:6813/1075185551","heartbeat_back_addr":"172.21.15.38:6817/1075185551","heartbeat_front_addr":"172.21.15.38:6815/1075185551","state":["exists","up"]},{"osd":2,"uuid":"cb38e881-abe9-449e-9ac6-ebe02bcfdfa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6819","nonce":1120523505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6821","nonce":1120523505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6825","nonce":1120523505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6823","nonce":1120523505}]},"public_addr":"172.21.15.38:6819/1120523505","cluster_addr":"172.21.15.38:6821/1120523505","heartbeat_back_addr":"172.21.15.38:6825/1120523505","heartbeat_front_addr":"172.21.15.38:6823/1120523505","state":["exists","up"]},{"osd":3,"uuid":"285121c6-325c-4c54-9fe3-a096a4acbf0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6827","nonce":2567848779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6829","nonce":2567848779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6833","nonce":2567848779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6831","nonce":2567848779}]},"public_addr":"172.21.15.38:6827/2567848779","cluster_addr":"172.21.15.38:6829/2567848779","heartbeat_back_addr":"172.21.15.38:6833/2567848779","heartbeat_front_addr":"172.21.15.38:6831/2567848779","state":["exists","up"]},{"osd":4,"uuid":"11ef947d-aea1-4d8e-a2e5-65fd221509a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051297666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051297666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051297666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051297666}]},"public_addr":"172.21.15.50:6801/2051297666","cluster_addr":"172.21.15.50:6803/2051297666","heartbeat_back_addr":"172.21.15.50:6807/2051297666","heartbeat_front_addr":"172.21.15.50:6805/2051297666","state":["exists","up"]},{"osd":5,"uuid":"c059913f-4e04-4414-b51b-f26db0e9f7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6809","nonce":765649226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6811","nonce":765649226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6815","nonce":765649226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6813","nonce":765649226}]},"public_addr":"172.21.15.50:6809/765649226","cluster_addr":"172.21.15.50:6811/765649226","heartbeat_back_addr":"172.21.15.50:6815/765649226","heartbeat_front_addr":"172.21.15.50:6813/765649226","state":["exists","up"]},{"osd":6,"uuid":"2c4169b6-e4b7-48ab-bf8c-4860d1c45b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6817","nonce":4014295619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6819","nonce":4014295619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6823","nonce":4014295619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6821","nonce":4014295619}]},"public_addr":"172.21.15.50:6817/4014295619","cluster_addr":"172.21.15.50:6819/4014295619","heartbeat_back_addr":"172.21.15.50:6823/4014295619","heartbeat_front_addr":"172.21.15.50:6821/4014295619","state":["exists","up"]},{"osd":7,"uuid":"0d3e6249-a211-42f4-95ee-03b1e56fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6825","nonce":4209821131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6827","nonce":4209821131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6831","nonce":4209821131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6829","nonce":4209821131}]},"public_addr":"172.21.15.50:6825/4209821131","cluster_addr":"172.21.15.50:6827/4209821131","heartbeat_back_addr":"172.21.15.50:6831/4209821131","heartbeat_front_addr":"172.21.15.50:6829/4209821131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:56:51.192855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:21.374997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:50.673882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:19.689087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:46.704082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:15.345782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:42.313545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:10.773193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6800/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/1200911576":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/3289402189":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/2044272138":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6801/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:6800/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1803280845":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/2740767068":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1465824294":"2024-09-11T03:55:00.513768+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:00:49.837 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:49 smithi050 ceph-mon[26687]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T04:00:49.837 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:49 smithi050 ceph-mon[26687]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T04:00:49.837 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:49 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3218577135' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:49.935 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T04:00:49.936 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T04:00:50.196 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:50.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[25393]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T04:00:50.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[25393]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T04:00:50.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3218577135' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:50.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[30767]: from='mgr.14152 172.21.15.38:0/2429631282' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T04:00:50.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[30767]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T04:00:50.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:49 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3218577135' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:50.885 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:00:50.885 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":70,"fsid":"3be9daae-6f28-11ef-bcea-c7b262605968","created":"2024-09-10T03:54:27.357206+0000","modified":"2024-09-10T04:00:30.377199+0000","last_up_change":"2024-09-10T04:00:12.324560+0000","last_in_change":"2024-09-10T03:59:48.127063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T03:57:53.950647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-10T04:00:20.228675+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":64,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-10T04:00:21.391089+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T04:00:23.155552+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T04:00:25.086993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T04:00:27.158517+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"808d5dc7-0ff8-439f-b0fa-93551f02dc05","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6803","nonce":4169351317}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6805","nonce":4169351317}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6809","nonce":4169351317}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":4169351317},{"type":"v1","addr":"172.21.15.38:6807","nonce":4169351317}]},"public_addr":"172.21.15.38:6803/4169351317","cluster_addr":"172.21.15.38:6805/4169351317","heartbeat_back_addr":"172.21.15.38:6809/4169351317","heartbeat_front_addr":"172.21.15.38:6807/4169351317","state":["exists","up"]},{"osd":1,"uuid":"83bc2578-eb39-48e2-8af2-c8214f4cd10d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6811","nonce":1075185551}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6813","nonce":1075185551}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6817","nonce":1075185551}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":1075185551},{"type":"v1","addr":"172.21.15.38:6815","nonce":1075185551}]},"public_addr":"172.21.15.38:6811/1075185551","cluster_addr":"172.21.15.38:6813/1075185551","heartbeat_back_addr":"172.21.15.38:6817/1075185551","heartbeat_front_addr":"172.21.15.38:6815/1075185551","state":["exists","up"]},{"osd":2,"uuid":"cb38e881-abe9-449e-9ac6-ebe02bcfdfa9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6819","nonce":1120523505}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6821","nonce":1120523505}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6825","nonce":1120523505}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1120523505},{"type":"v1","addr":"172.21.15.38:6823","nonce":1120523505}]},"public_addr":"172.21.15.38:6819/1120523505","cluster_addr":"172.21.15.38:6821/1120523505","heartbeat_back_addr":"172.21.15.38:6825/1120523505","heartbeat_front_addr":"172.21.15.38:6823/1120523505","state":["exists","up"]},{"osd":3,"uuid":"285121c6-325c-4c54-9fe3-a096a4acbf0a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6827","nonce":2567848779}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6829","nonce":2567848779}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6833","nonce":2567848779}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":2567848779},{"type":"v1","addr":"172.21.15.38:6831","nonce":2567848779}]},"public_addr":"172.21.15.38:6827/2567848779","cluster_addr":"172.21.15.38:6829/2567848779","heartbeat_back_addr":"172.21.15.38:6833/2567848779","heartbeat_front_addr":"172.21.15.38:6831/2567848779","state":["exists","up"]},{"osd":4,"uuid":"11ef947d-aea1-4d8e-a2e5-65fd221509a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6800","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6801","nonce":2051297666}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6802","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6803","nonce":2051297666}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6806","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6807","nonce":2051297666}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6804","nonce":2051297666},{"type":"v1","addr":"172.21.15.50:6805","nonce":2051297666}]},"public_addr":"172.21.15.50:6801/2051297666","cluster_addr":"172.21.15.50:6803/2051297666","heartbeat_back_addr":"172.21.15.50:6807/2051297666","heartbeat_front_addr":"172.21.15.50:6805/2051297666","state":["exists","up"]},{"osd":5,"uuid":"c059913f-4e04-4414-b51b-f26db0e9f7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6808","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6809","nonce":765649226}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6810","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6811","nonce":765649226}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6814","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6815","nonce":765649226}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6812","nonce":765649226},{"type":"v1","addr":"172.21.15.50:6813","nonce":765649226}]},"public_addr":"172.21.15.50:6809/765649226","cluster_addr":"172.21.15.50:6811/765649226","heartbeat_back_addr":"172.21.15.50:6815/765649226","heartbeat_front_addr":"172.21.15.50:6813/765649226","state":["exists","up"]},{"osd":6,"uuid":"2c4169b6-e4b7-48ab-bf8c-4860d1c45b25","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6816","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6817","nonce":4014295619}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6818","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6819","nonce":4014295619}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6822","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6823","nonce":4014295619}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6820","nonce":4014295619},{"type":"v1","addr":"172.21.15.50:6821","nonce":4014295619}]},"public_addr":"172.21.15.50:6817/4014295619","cluster_addr":"172.21.15.50:6819/4014295619","heartbeat_back_addr":"172.21.15.50:6823/4014295619","heartbeat_front_addr":"172.21.15.50:6821/4014295619","state":["exists","up"]},{"osd":7,"uuid":"0d3e6249-a211-42f4-95ee-03b1e56fc521","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6824","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6825","nonce":4209821131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6826","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6827","nonce":4209821131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6830","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6831","nonce":4209821131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.50:6828","nonce":4209821131},{"type":"v1","addr":"172.21.15.50:6829","nonce":4209821131}]},"public_addr":"172.21.15.50:6825/4209821131","cluster_addr":"172.21.15.50:6827/4209821131","heartbeat_back_addr":"172.21.15.50:6831/4209821131","heartbeat_front_addr":"172.21.15.50:6829/4209821131","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:56:51.192855+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:21.374997+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:57:50.673882+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:19.689087+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:58:46.704082+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:15.345782+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T03:59:42.313545+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T04:00:10.773193+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6800/3987899242":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/1200911576":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/3289402189":"2024-09-11T03:55:27.476860+0000","172.21.15.38:0/2044272138":"2024-09-11T03:55:27.476860+0000","172.21.15.38:6801/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:6800/3732646376":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1803280845":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/2740767068":"2024-09-11T03:55:00.513768+0000","172.21.15.38:0/1465824294":"2024-09-11T03:55:00.513768+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T04:00:51.043 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:50 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2031459926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:51.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:50 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2031459926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:51.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:50 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2031459926' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-10T04:00:51.685 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T04:00:51.686 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T04:00:51.687 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T04:00:51.687 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T04:00:51.688 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T04:00:51.689 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T04:00:51.689 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T04:00:51.690 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T04:00:52.237 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:52.237 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:52.237 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:52.272 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:52.272 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:54.008 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:54.384 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: Standby manager daemon x restarted 2024-09-10T04:00:54.384 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: Standby manager daemon x started 2024-09-10T04:00:54.384 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T04:00:54.384 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[30767]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: Standby manager daemon x restarted 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: Standby manager daemon x started 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T04:00:54.385 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:54 smithi038 ceph-mon[25393]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T04:00:54.528 INFO:teuthology.orchestra.run.smithi038.stdout:111669149729 2024-09-10T04:00:54.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: Standby manager daemon x restarted 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: Standby manager daemon x started 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/crt"} : dispatch 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/x/key"} : dispatch 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-mon[26687]: from='mgr.? 172.21.15.50:0/1552668820' entity='mgr.x' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: [10/Sep/2024:04:00:54] ENGINE Bus STARTING 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: CherryPy Checker: 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: The Application mounted at '' has an empty config. 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: 2024-09-10T04:00:54.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: [10/Sep/2024:04:00:54] ENGINE Serving on http://:::9283 2024-09-10T04:00:54.556 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:00:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x[28554]: [10/Sep/2024:04:00:54] ENGINE Bus STARTED 2024-09-10T04:00:54.641 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:55.460 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: Active manager daemon y restarted 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: Activating manager daemon y 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: mgrmap e20: y(active, starting, since 0.0132039s), standbys: x 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T04:00:55.461 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T04:00:55.462 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T04:00:55.463 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T04:00:55.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T04:00:55.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: Manager daemon y is now available 2024-09-10T04:00:55.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.464 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:55.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.465 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.466 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: Active manager daemon y restarted 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: Activating manager daemon y 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T04:00:55.467 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: mgrmap e20: y(active, starting, since 0.0132039s), standbys: x 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:55.468 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: Manager daemon y is now available 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.469 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: mgrmap e19: y(active, since 5m), standbys: x 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: Active manager daemon y restarted 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: Activating manager daemon y 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: mgrmap e20: y(active, starting, since 0.0132039s), standbys: x 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-10T04:00:55.470 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mds metadata"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd metadata"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "mon metadata"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: Manager daemon y is now available 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.471 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"} : dispatch 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi050", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:00:55.472 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]': finished 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:00:55.473 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.474 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"} : dispatch 2024-09-10T04:00:55.474 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.474 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"} : dispatch 2024-09-10T04:00:55.743 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:00:55] ENGINE Bus STARTING 2024-09-10T04:00:55.743 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: CherryPy Checker: 2024-09-10T04:00:55.743 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: The Application mounted at '' has an empty config. 2024-09-10T04:00:55.743 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T04:00:55.743 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:00:55] ENGINE Serving on http://:::9283 2024-09-10T04:00:55.744 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:55 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:00:55] ENGINE Bus STARTED 2024-09-10T04:00:55.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:00:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:00:56.002 INFO:teuthology.orchestra.run.smithi038.stdout:34359738418 2024-09-10T04:00:56.003 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T04:00:56.221 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:00:56.627 INFO:teuthology.orchestra.run.smithi038.stdout:236223201292 2024-09-10T04:00:56.628 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T04:00:56.650 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: mgrmap e21: y(active, since 1.02552s), standbys: x 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.651 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: mgrmap e21: y(active, since 1.02552s), standbys: x 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:56.652 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi050:/etc/ceph/ceph.conf 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: mgrmap e21: y(active, since 1.02552s), standbys: x 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: pgmap v3: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:56.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: Updating smithi050:/etc/ceph/ceph.client.admin.keyring 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:56.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:56 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:00:57.004 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T04:00:56.659+0000 7fa748be7640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:04:00:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: Traceback (most recent call last): 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._connections.run(self.expiration_interval) 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._run(expiration_interval) 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: s = self.context.wrap_socket( 2024-09-10T04:00:57.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: return self.sslsocket_class._create( 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self.do_handshake() 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: self._sslobj.do_handshake() 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:00:57.006 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:00:56 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T04:00:57.107 INFO:teuthology.orchestra.run.smithi038.stdout:167503724566 2024-09-10T04:00:57.107 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T04:00:57.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: Deploying daemon alertmanager.a on smithi038 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: [10/Sep/2024:04:00:56] ENGINE Bus STARTING 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: [10/Sep/2024:04:00:56] ENGINE Serving on http://172.21.15.38:8765 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: [10/Sep/2024:04:00:56] ENGINE Serving on https://172.21.15.38:7151 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: [10/Sep/2024:04:00:56] ENGINE Bus STARTED 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: [10/Sep/2024:04:00:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout: Traceback (most recent call last): 2024-09-10T04:00:57.755 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: self._run(expiration_interval) 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: s = self.context.wrap_socket( 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: return self.sslsocket_class._create( 2024-09-10T04:00:57.756 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.c.smithi038.stdout: self.do_handshake() 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.c.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.c.smithi038.stdout: self._sslobj.do_handshake() 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.c.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[30767]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: Deploying daemon alertmanager.a on smithi038 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: [10/Sep/2024:04:00:56] ENGINE Bus STARTING 2024-09-10T04:00:57.757 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: [10/Sep/2024:04:00:56] ENGINE Serving on http://172.21.15.38:8765 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: [10/Sep/2024:04:00:56] ENGINE Serving on https://172.21.15.38:7151 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: [10/Sep/2024:04:00:56] ENGINE Bus STARTED 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: [10/Sep/2024:04:00:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:00:57.758 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:00:57.759 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:57 smithi038 ceph-mon[25393]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: Updating smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: Updating smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: Deploying daemon alertmanager.a on smithi038 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: [10/Sep/2024:04:00:56] ENGINE Bus STARTING 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: [10/Sep/2024:04:00:56] ENGINE Serving on http://172.21.15.38:8765 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: [10/Sep/2024:04:00:56] ENGINE Serving on https://172.21.15.38:7151 2024-09-10T04:00:57.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: [10/Sep/2024:04:00:56] ENGINE Bus STARTED 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: [10/Sep/2024:04:00:56] ENGINE Error in HTTPServer.serve 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: Traceback (most recent call last): 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._connections.run(self.expiration_interval) 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._run(expiration_interval) 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: s = self.context.wrap_socket( 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T04:00:57.806 INFO:journalctl@ceph.mon.b.smithi050.stdout: return self.sslsocket_class._create( 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout: self.do_handshake() 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout: self._sslobj.do_handshake() 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T04:00:57.807 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:57 smithi050 ceph-mon[26687]: mgrmap e22: y(active, since 2s), standbys: x 2024-09-10T04:00:59.556 INFO:teuthology.orchestra.run.smithi038.stdout:133143986203 2024-09-10T04:00:59.557 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T04:00:59.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:59 smithi050 ceph-mon[26687]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:59.746 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:00:59 smithi050 ceph-mon[26687]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-10T04:00:59.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:59 smithi038 ceph-mon[25393]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:59.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:00:59 smithi038 ceph-mon[25393]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-10T04:00:59.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:59 smithi038 ceph-mon[30767]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:00:59.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:00:59 smithi038 ceph-mon[30767]: mgrmap e23: y(active, since 4s), standbys: x 2024-09-10T04:01:00.188 INFO:teuthology.orchestra.run.smithi038.stdout:77309411368 2024-09-10T04:01:00.188 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T04:01:00.400 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:00.404 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:00.917 INFO:teuthology.orchestra.run.smithi038.stdout:201863462929 2024-09-10T04:01:00.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T04:01:01.411 INFO:teuthology.orchestra.run.smithi038.stdout:55834574893 2024-09-10T04:01:01.412 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T04:01:01.433 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:01.436 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:01.610 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:01 smithi038 ceph-mon[30767]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:01:01.611 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:01 smithi038 ceph-mon[25393]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:01:01.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:01 smithi050 ceph-mon[26687]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-09-10T04:01:02.341 INFO:teuthology.orchestra.run.smithi038.stdout:77309411368 2024-09-10T04:01:02.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:02 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3011308413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T04:01:02.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:02 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3011308413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T04:01:02.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:02 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3011308413' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-10T04:01:03.247 INFO:teuthology.orchestra.run.smithi038.stdout:133143986204 2024-09-10T04:01:03.540 INFO:teuthology.orchestra.run.smithi038.stdout:34359738420 2024-09-10T04:01:03.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:03 smithi038 ceph-mon[30767]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T04:01:03.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:03 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2460974605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T04:01:03.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:03 smithi038 ceph-mon[25393]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T04:01:03.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:03 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2460974605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T04:01:03.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:03 smithi050 ceph-mon[26687]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T04:01:03.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:03 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2460974605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-10T04:01:04.504 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:04 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ::ffff:172.21.15.50 - - [10/Sep/2024:04:01:04] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-10T04:01:04.625 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986203 got 133143986204 for osd.4 2024-09-10T04:01:04.625 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:04.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:04 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3124508613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T04:01:04.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:04 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3124508613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T04:01:04.964 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:04 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3124508613' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-10T04:01:05.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:05 smithi038 ceph-mon[25393]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T04:01:05.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:05 smithi038 ceph-mon[30767]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T04:01:05.763 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:05 smithi050 ceph-mon[26687]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T04:01:05.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:01:05 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:01:06.571 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411368 for osd.2 2024-09-10T04:01:06.571 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:06.796 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:06 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:06.796 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:06 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:06.797 INFO:teuthology.orchestra.run.smithi038.stdout:111669149732 2024-09-10T04:01:06.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:06 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:06.857 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738418 got 34359738420 for osd.0 2024-09-10T04:01:06.857 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:07.307 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:07.310 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:07.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:07 smithi038 ceph-mon[25393]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:07.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:07 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/3639041442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:07 smithi038 ceph-mon[30767]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:07.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:07 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/3639041442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T04:01:07.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:07 smithi050 ceph-mon[26687]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:07.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:07 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/3639041442' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-10T04:01:07.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149729 got 111669149732 for osd.3 2024-09-10T04:01:07.872 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:07.899 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:07.899 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:08.879 INFO:teuthology.orchestra.run.smithi038.stdout:167503724569 2024-09-10T04:01:09.182 INFO:teuthology.orchestra.run.smithi038.stdout:236223201294 2024-09-10T04:01:09.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[25393]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:09.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2002761182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T04:01:09.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/453140109' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T04:01:09.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[30767]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:09.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2002761182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T04:01:09.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:09 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/453140109' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T04:01:09.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:09 smithi050 ceph-mon[26687]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:09 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2002761182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-10T04:01:09.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:09 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/453140109' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-10T04:01:10.658 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:10 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:10.658 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:10 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:10.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:10 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:11.742 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:11 smithi038 ceph-mon[25393]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:11.742 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:11 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/1823591268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T04:01:11.742 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:11 smithi038 ceph-mon[30767]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:11.742 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:11 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/1823591268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T04:01:11.777 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201292 got 236223201294 for osd.7 2024-09-10T04:01:11.778 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:12.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:11 smithi050 ceph-mon[26687]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T04:01:12.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:11 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/1823591268' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-10T04:01:12.103 INFO:teuthology.orchestra.run.smithi038.stdout:55834574895 2024-09-10T04:01:12.721 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574893 got 55834574895 for osd.1 2024-09-10T04:01:12.721 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:13.462 INFO:teuthology.orchestra.run.smithi038.stdout:201863462932 2024-09-10T04:01:13.731 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:13 smithi038 ceph-mon[25393]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:13.731 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:13 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/2271413803' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T04:01:13.732 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:13 smithi038 ceph-mon[30767]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:13.732 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:13 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/2271413803' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T04:01:14.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:13 smithi050 ceph-mon[26687]: pgmap v12: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T04:01:14.117 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:13 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/2271413803' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-10T04:01:14.360 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:14 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ::ffff:172.21.15.50 - - [10/Sep/2024:04:01:14] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-10T04:01:15.685 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:15 smithi038 ceph-mon[25393]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:15.685 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:15 smithi038 ceph-mon[30767]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:15.804 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:15 smithi050 ceph-mon[26687]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:15.805 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:01:15 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:01:16.255 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:16 smithi038 systemd[1]: Starting Ceph alertmanager.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:16.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462929 got 201863462932 for osd.6 2024-09-10T04:01:16.434 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:16.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:16 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:16.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:16 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:16.848 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724566 got 167503724569 for osd.5 2024-09-10T04:01:16.848 DEBUG:teuthology.parallel:result is None 2024-09-10T04:01:16.848 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T04:01:16.849 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T04:01:17.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:16 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:17.119 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 podman[55366]: 2024-09-10 04:01:17.017906654 +0000 UTC m=+0.108503469 volume create e84156c85523826948b545bfe949430bf8260aaff3910ff211655c54f23f864f 2024-09-10T04:01:17.119 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 podman[55366]: 2024-09-10 04:01:16.926827865 +0000 UTC m=+0.017424688 image pull 11f11916f8cdfb17c1da74b20deb2075d778357b4d60474d82453303f8d461f5 quay.io/prometheus/alertmanager:v0.27.0 2024-09-10T04:01:17.119 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 podman[55366]: 2024-09-10 04:01:17.118386433 +0000 UTC m=+0.208983247 container create 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:17.580 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:17.703 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-mon[25393]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:17.704 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-mon[30767]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 podman[55366]: 2024-09-10 04:01:17.446321677 +0000 UTC m=+0.536918494 container init 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 podman[55366]: 2024-09-10 04:01:17.449550634 +0000 UTC m=+0.540147457 container start 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.459Z caller=main.go:181 level=info msg="Starting Alertmanager" version="(version=0.27.0, branch=HEAD, revision=0aa3c2aad14cff039931923ab16b26b7481783b5)" 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.460Z caller=main.go:182 level=info build_context="(go=go1.21.7, platform=linux/amd64, user=root@22cd11f671e9, date=20240228-11:51:20, tags=netgo)" 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.461Z caller=cluster.go:186 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.38 port=9094 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.462Z caller=cluster.go:683 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.483Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T04:01:17.704 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.484Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T04:01:17.705 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.486Z caller=tls_config.go:313 level=info msg="Listening on" address=[::]:9093 2024-09-10T04:01:17.705 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:17.486Z caller=tls_config.go:316 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T04:01:17.705 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 bash[55366]: 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 2024-09-10T04:01:17.705 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:17 smithi038 systemd[1]: Started Ceph alertmanager.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:01:18.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:17 smithi050 ceph-mon[26687]: pgmap v14: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:18.257 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE Bus STOPPING 2024-09-10T04:01:18.258 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T04:01:18.258 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE Bus STOPPED 2024-09-10T04:01:18.258 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE Bus STARTING 2024-09-10T04:01:18.345 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:01:18.347 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-10T04:01:18.607 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE Serving on http://:::9283 2024-09-10T04:01:18.607 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:18] ENGINE Bus STARTED 2024-09-10T04:01:18.957 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:18.958 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:18 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"} : dispatch 2024-09-10T04:01:19.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:19.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:18 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "orch prometheus get-credentials"} : dispatch 2024-09-10T04:01:19.355 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-09-10T04:01:16.458079+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":231,"num_object_clones":0,"num_object_copies":693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":1213,"num_read_kb":1035,"num_write":807,"num_write_kb":3239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":805,"ondisk_log_size":805,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":236656,"kb_used_data":7964,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749691792,"statfs":{"total":767926730752,"available":767684395008,"internally_reserved":0,"allocated":8155136,"data_stored":4381809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":234016307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002987"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640953+0000","last_change":"2024-09-10T04:00:23.072686+0000","last_active":"2024-09-10T04:00:54.640953+0000","last_peered":"2024-09-10T04:00:54.640953+0000","last_clean":"2024-09-10T04:00:54.640953+0000","last_became_active":"2024-09-10T04:00:23.072412+0000","last_became_peered":"2024-09-10T04:00:23.072412+0000","last_unstale":"2024-09-10T04:00:54.640953+0000","last_undegraded":"2024-09-10T04:00:54.640953+0000","last_fullsized":"2024-09-10T04:00:54.640953+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:35.269691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640446+0000","last_change":"2024-09-10T04:00:25.076327+0000","last_active":"2024-09-10T04:00:54.640446+0000","last_peered":"2024-09-10T04:00:54.640446+0000","last_clean":"2024-09-10T04:00:54.640446+0000","last_became_active":"2024-09-10T04:00:25.076009+0000","last_became_peered":"2024-09-10T04:00:25.076009+0000","last_unstale":"2024-09-10T04:00:54.640446+0000","last_undegraded":"2024-09-10T04:00:54.640446+0000","last_fullsized":"2024-09-10T04:00:54.640446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:30:02.311326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641396+0000","last_change":"2024-09-10T04:00:27.105165+0000","last_active":"2024-09-10T04:00:54.641396+0000","last_peered":"2024-09-10T04:00:54.641396+0000","last_clean":"2024-09-10T04:00:54.641396+0000","last_became_active":"2024-09-10T04:00:27.104904+0000","last_became_peered":"2024-09-10T04:00:27.104904+0000","last_unstale":"2024-09-10T04:00:54.641396+0000","last_undegraded":"2024-09-10T04:00:54.641396+0000","last_fullsized":"2024-09-10T04:00:54.641396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:19:21.256230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640612+0000","last_change":"2024-09-10T04:00:29.134903+0000","last_active":"2024-09-10T04:00:54.640612+0000","last_peered":"2024-09-10T04:00:54.640612+0000","last_clean":"2024-09-10T04:00:54.640612+0000","last_became_active":"2024-09-10T04:00:29.134519+0000","last_became_peered":"2024-09-10T04:00:29.134519+0000","last_unstale":"2024-09-10T04:00:54.640612+0000","last_undegraded":"2024-09-10T04:00:54.640612+0000","last_fullsized":"2024-09-10T04:00:54.640612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:12.176160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441613+0000","last_change":"2024-09-10T04:00:29.246054+0000","last_active":"2024-09-10T04:00:54.441613+0000","last_peered":"2024-09-10T04:00:54.441613+0000","last_clean":"2024-09-10T04:00:54.441613+0000","last_became_active":"2024-09-10T04:00:29.245687+0000","last_became_peered":"2024-09-10T04:00:29.245687+0000","last_unstale":"2024-09-10T04:00:54.441613+0000","last_undegraded":"2024-09-10T04:00:54.441613+0000","last_fullsized":"2024-09-10T04:00:54.441613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:28.145580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442454+0000","last_change":"2024-09-10T04:00:23.078699+0000","last_active":"2024-09-10T04:00:54.442454+0000","last_peered":"2024-09-10T04:00:54.442454+0000","last_clean":"2024-09-10T04:00:54.442454+0000","last_became_active":"2024-09-10T04:00:23.078396+0000","last_became_peered":"2024-09-10T04:00:23.078396+0000","last_unstale":"2024-09-10T04:00:54.442454+0000","last_undegraded":"2024-09-10T04:00:54.442454+0000","last_fullsized":"2024-09-10T04:00:54.442454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:31.182539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'39","reported_seq":99,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441650+0000","last_change":"2024-09-10T04:00:25.073606+0000","last_active":"2024-09-10T04:00:54.441650+0000","last_peered":"2024-09-10T04:00:54.441650+0000","last_clean":"2024-09-10T04:00:54.441650+0000","last_became_active":"2024-09-10T04:00:25.073287+0000","last_became_peered":"2024-09-10T04:00:25.073287+0000","last_unstale":"2024-09-10T04:00:54.441650+0000","last_undegraded":"2024-09-10T04:00:54.441650+0000","last_fullsized":"2024-09-10T04:00:54.441650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:56.652958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":62,"num_read_kb":49,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640358+0000","last_change":"2024-09-10T04:00:27.102876+0000","last_active":"2024-09-10T04:00:54.640358+0000","last_peered":"2024-09-10T04:00:54.640358+0000","last_clean":"2024-09-10T04:00:54.640358+0000","last_became_active":"2024-09-10T04:00:27.102604+0000","last_became_peered":"2024-09-10T04:00:27.102604+0000","last_unstale":"2024-09-10T04:00:54.640358+0000","last_undegraded":"2024-09-10T04:00:54.640358+0000","last_fullsized":"2024-09-10T04:00:54.640358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:59.508066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640171+0000","last_change":"2024-09-10T04:00:23.060679+0000","last_active":"2024-09-10T04:00:54.640171+0000","last_peered":"2024-09-10T04:00:54.640171+0000","last_clean":"2024-09-10T04:00:54.640171+0000","last_became_active":"2024-09-10T04:00:23.060219+0000","last_became_peered":"2024-09-10T04:00:23.060219+0000","last_unstale":"2024-09-10T04:00:54.640171+0000","last_undegraded":"2024-09-10T04:00:54.640171+0000","last_fullsized":"2024-09-10T04:00:54.640171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:13:20.124081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'21","reported_seq":66,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640742+0000","last_change":"2024-09-10T04:00:25.070774+0000","last_active":"2024-09-10T04:00:54.640742+0000","last_peered":"2024-09-10T04:00:54.640742+0000","last_clean":"2024-09-10T04:00:54.640742+0000","last_became_active":"2024-09-10T04:00:25.070422+0000","last_became_peered":"2024-09-10T04:00:25.070422+0000","last_unstale":"2024-09-10T04:00:54.640742+0000","last_undegraded":"2024-09-10T04:00:54.640742+0000","last_fullsized":"2024-09-10T04:00:54.640742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:11.044787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638765+0000","last_change":"2024-09-10T04:00:27.103057+0000","last_active":"2024-09-10T04:00:54.638765+0000","last_peered":"2024-09-10T04:00:54.638765+0000","last_clean":"2024-09-10T04:00:54.638765+0000","last_became_active":"2024-09-10T04:00:27.102772+0000","last_became_peered":"2024-09-10T04:00:27.102772+0000","last_unstale":"2024-09-10T04:00:54.638765+0000","last_undegraded":"2024-09-10T04:00:54.638765+0000","last_fullsized":"2024-09-10T04:00:54.638765+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:56:04.069894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639588+0000","last_change":"2024-09-10T04:00:29.134358+0000","last_active":"2024-09-10T04:00:54.639588+0000","last_peered":"2024-09-10T04:00:54.639588+0000","last_clean":"2024-09-10T04:00:54.639588+0000","last_became_active":"2024-09-10T04:00:29.133979+0000","last_became_peered":"2024-09-10T04:00:29.133979+0000","last_unstale":"2024-09-10T04:00:54.639588+0000","last_undegraded":"2024-09-10T04:00:54.639588+0000","last_fullsized":"2024-09-10T04:00:54.639588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:35.904688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639509+0000","last_change":"2024-09-10T04:00:23.076834+0000","last_active":"2024-09-10T04:00:54.639509+0000","last_peered":"2024-09-10T04:00:54.639509+0000","last_clean":"2024-09-10T04:00:54.639509+0000","last_became_active":"2024-09-10T04:00:23.076398+0000","last_became_peered":"2024-09-10T04:00:23.076398+0000","last_unstale":"2024-09-10T04:00:54.639509+0000","last_undegraded":"2024-09-10T04:00:54.639509+0000","last_fullsized":"2024-09-10T04:00:54.639509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:39.606592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640670+0000","last_change":"2024-09-10T04:00:25.079520+0000","last_active":"2024-09-10T04:00:54.640670+0000","last_peered":"2024-09-10T04:00:54.640670+0000","last_clean":"2024-09-10T04:00:54.640670+0000","last_became_active":"2024-09-10T04:00:25.079006+0000","last_became_peered":"2024-09-10T04:00:25.079006+0000","last_unstale":"2024-09-10T04:00:54.640670+0000","last_undegraded":"2024-09-10T04:00:54.640670+0000","last_fullsized":"2024-09-10T04:00:54.640670+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:59.277084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":15,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438211+0000","last_change":"2024-09-10T04:00:27.105721+0000","last_active":"2024-09-10T04:00:54.438211+0000","last_peered":"2024-09-10T04:00:54.438211+0000","last_clean":"2024-09-10T04:00:54.438211+0000","last_became_active":"2024-09-10T04:00:27.105431+0000","last_became_peered":"2024-09-10T04:00:27.105431+0000","last_unstale":"2024-09-10T04:00:54.438211+0000","last_undegraded":"2024-09-10T04:00:54.438211+0000","last_fullsized":"2024-09-10T04:00:54.438211+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:44.569653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638664+0000","last_change":"2024-09-10T04:00:29.129999+0000","last_active":"2024-09-10T04:00:54.638664+0000","last_peered":"2024-09-10T04:00:54.638664+0000","last_clean":"2024-09-10T04:00:54.638664+0000","last_became_active":"2024-09-10T04:00:29.129706+0000","last_became_peered":"2024-09-10T04:00:29.129706+0000","last_unstale":"2024-09-10T04:00:54.638664+0000","last_undegraded":"2024-09-10T04:00:54.638664+0000","last_fullsized":"2024-09-10T04:00:54.638664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:37.456263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641438+0000","last_change":"2024-09-10T04:00:29.245498+0000","last_active":"2024-09-10T04:00:54.641438+0000","last_peered":"2024-09-10T04:00:54.641438+0000","last_clean":"2024-09-10T04:00:54.641438+0000","last_became_active":"2024-09-10T04:00:29.245161+0000","last_became_peered":"2024-09-10T04:00:29.245161+0000","last_unstale":"2024-09-10T04:00:54.641438+0000","last_undegraded":"2024-09-10T04:00:54.641438+0000","last_fullsized":"2024-09-10T04:00:54.641438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:15.406497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.644046+0000","last_change":"2024-09-10T04:00:23.070438+0000","last_active":"2024-09-10T04:00:54.644046+0000","last_peered":"2024-09-10T04:00:54.644046+0000","last_clean":"2024-09-10T04:00:54.644046+0000","last_became_active":"2024-09-10T04:00:23.070085+0000","last_became_peered":"2024-09-10T04:00:23.070085+0000","last_unstale":"2024-09-10T04:00:54.644046+0000","last_undegraded":"2024-09-10T04:00:54.644046+0000","last_fullsized":"2024-09-10T04:00:54.644046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:38.781922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447238+0000","last_change":"2024-09-10T04:00:25.077613+0000","last_active":"2024-09-10T04:00:54.447238+0000","last_peered":"2024-09-10T04:00:54.447238+0000","last_clean":"2024-09-10T04:00:54.447238+0000","last_became_active":"2024-09-10T04:00:25.077282+0000","last_became_peered":"2024-09-10T04:00:25.077282+0000","last_unstale":"2024-09-10T04:00:54.447238+0000","last_undegraded":"2024-09-10T04:00:54.447238+0000","last_fullsized":"2024-09-10T04:00:54.447238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:37:44.850925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642848+0000","last_change":"2024-09-10T04:00:27.106867+0000","last_active":"2024-09-10T04:00:54.642848+0000","last_peered":"2024-09-10T04:00:54.642848+0000","last_clean":"2024-09-10T04:00:54.642848+0000","last_became_active":"2024-09-10T04:00:27.106419+0000","last_became_peered":"2024-09-10T04:00:27.106419+0000","last_unstale":"2024-09-10T04:00:54.642848+0000","last_undegraded":"2024-09-10T04:00:54.642848+0000","last_fullsized":"2024-09-10T04:00:54.642848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:33.749714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442334+0000","last_change":"2024-09-10T04:00:29.245925+0000","last_active":"2024-09-10T04:00:54.442334+0000","last_peered":"2024-09-10T04:00:54.442334+0000","last_clean":"2024-09-10T04:00:54.442334+0000","last_became_active":"2024-09-10T04:00:29.245504+0000","last_became_peered":"2024-09-10T04:00:29.245504+0000","last_unstale":"2024-09-10T04:00:54.442334+0000","last_undegraded":"2024-09-10T04:00:54.442334+0000","last_fullsized":"2024-09-10T04:00:54.442334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:13.293950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640900+0000","last_change":"2024-09-10T04:00:23.080555+0000","last_active":"2024-09-10T04:00:54.640900+0000","last_peered":"2024-09-10T04:00:54.640900+0000","last_clean":"2024-09-10T04:00:54.640900+0000","last_became_active":"2024-09-10T04:00:23.079901+0000","last_became_peered":"2024-09-10T04:00:23.079901+0000","last_unstale":"2024-09-10T04:00:54.640900+0000","last_undegraded":"2024-09-10T04:00:54.640900+0000","last_fullsized":"2024-09-10T04:00:54.640900+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:02.017897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442124+0000","last_change":"2024-09-10T04:00:25.076449+0000","last_active":"2024-09-10T04:00:54.442124+0000","last_peered":"2024-09-10T04:00:54.442124+0000","last_clean":"2024-09-10T04:00:54.442124+0000","last_became_active":"2024-09-10T04:00:25.076116+0000","last_became_peered":"2024-09-10T04:00:25.076116+0000","last_unstale":"2024-09-10T04:00:54.442124+0000","last_undegraded":"2024-09-10T04:00:54.442124+0000","last_fullsized":"2024-09-10T04:00:54.442124+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:58:29.508435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640746+0000","last_change":"2024-09-10T04:00:27.106013+0000","last_active":"2024-09-10T04:00:54.640746+0000","last_peered":"2024-09-10T04:00:54.640746+0000","last_clean":"2024-09-10T04:00:54.640746+0000","last_became_active":"2024-09-10T04:00:27.105741+0000","last_became_peered":"2024-09-10T04:00:27.105741+0000","last_unstale":"2024-09-10T04:00:54.640746+0000","last_undegraded":"2024-09-10T04:00:54.640746+0000","last_fullsized":"2024-09-10T04:00:54.640746+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:20:21.840639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438962+0000","last_change":"2024-09-10T04:00:29.136695+0000","last_active":"2024-09-10T04:00:54.438962+0000","last_peered":"2024-09-10T04:00:54.438962+0000","last_clean":"2024-09-10T04:00:54.438962+0000","last_became_active":"2024-09-10T04:00:29.136386+0000","last_became_peered":"2024-09-10T04:00:29.136386+0000","last_unstale":"2024-09-10T04:00:54.438962+0000","last_undegraded":"2024-09-10T04:00:54.438962+0000","last_fullsized":"2024-09-10T04:00:54.438962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:38.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641580+0000","last_change":"2024-09-10T04:00:23.074974+0000","last_active":"2024-09-10T04:00:54.641580+0000","last_peered":"2024-09-10T04:00:54.641580+0000","last_clean":"2024-09-10T04:00:54.641580+0000","last_became_active":"2024-09-10T04:00:23.074610+0000","last_became_peered":"2024-09-10T04:00:23.074610+0000","last_unstale":"2024-09-10T04:00:54.641580+0000","last_undegraded":"2024-09-10T04:00:54.641580+0000","last_fullsized":"2024-09-10T04:00:54.641580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:01:08.194672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'16","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642143+0000","last_change":"2024-09-10T04:00:25.186355+0000","last_active":"2024-09-10T04:00:54.642143+0000","last_peered":"2024-09-10T04:00:54.642143+0000","last_clean":"2024-09-10T04:00:54.642143+0000","last_became_active":"2024-09-10T04:00:25.185910+0000","last_became_peered":"2024-09-10T04:00:25.185910+0000","last_unstale":"2024-09-10T04:00:54.642143+0000","last_undegraded":"2024-09-10T04:00:54.642143+0000","last_fullsized":"2024-09-10T04:00:54.642143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:43.441887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137826+0000","last_change":"2024-09-10T04:00:27.104212+0000","last_active":"2024-09-10T04:01:08.137826+0000","last_peered":"2024-09-10T04:01:08.137826+0000","last_clean":"2024-09-10T04:01:08.137826+0000","last_became_active":"2024-09-10T04:00:27.103921+0000","last_became_peered":"2024-09-10T04:00:27.103921+0000","last_unstale":"2024-09-10T04:01:08.137826+0000","last_undegraded":"2024-09-10T04:01:08.137826+0000","last_fullsized":"2024-09-10T04:01:08.137826+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:38.029153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643396+0000","last_change":"2024-09-10T04:00:25.080380+0000","last_active":"2024-09-10T04:00:54.643396+0000","last_peered":"2024-09-10T04:00:54.643396+0000","last_clean":"2024-09-10T04:00:54.643396+0000","last_became_active":"2024-09-10T04:00:25.080076+0000","last_became_peered":"2024-09-10T04:00:25.080076+0000","last_unstale":"2024-09-10T04:00:54.643396+0000","last_undegraded":"2024-09-10T04:00:54.643396+0000","last_fullsized":"2024-09-10T04:00:54.643396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:48.997357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441736+0000","last_change":"2024-09-10T04:00:23.076719+0000","last_active":"2024-09-10T04:00:54.441736+0000","last_peered":"2024-09-10T04:00:54.441736+0000","last_clean":"2024-09-10T04:00:54.441736+0000","last_became_active":"2024-09-10T04:00:23.076027+0000","last_became_peered":"2024-09-10T04:00:23.076027+0000","last_unstale":"2024-09-10T04:00:54.441736+0000","last_undegraded":"2024-09-10T04:00:54.441736+0000","last_fullsized":"2024-09-10T04:00:54.441736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:04.555840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137635+0000","last_change":"2024-09-10T04:00:27.102303+0000","last_active":"2024-09-10T04:01:08.137635+0000","last_peered":"2024-09-10T04:01:08.137635+0000","last_clean":"2024-09-10T04:01:08.137635+0000","last_became_active":"2024-09-10T04:00:27.101831+0000","last_became_peered":"2024-09-10T04:00:27.101831+0000","last_unstale":"2024-09-10T04:01:08.137635+0000","last_undegraded":"2024-09-10T04:01:08.137635+0000","last_fullsized":"2024-09-10T04:01:08.137635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:20.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639212+0000","last_change":"2024-09-10T04:00:29.134284+0000","last_active":"2024-09-10T04:00:54.639212+0000","last_peered":"2024-09-10T04:00:54.639212+0000","last_clean":"2024-09-10T04:00:54.639212+0000","last_became_active":"2024-09-10T04:00:29.133969+0000","last_became_peered":"2024-09-10T04:00:29.133969+0000","last_unstale":"2024-09-10T04:00:54.639212+0000","last_undegraded":"2024-09-10T04:00:54.639212+0000","last_fullsized":"2024-09-10T04:00:54.639212+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:30.203485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"70'30","reported_seq":82,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441417+0000","last_change":"2024-09-10T04:00:25.186762+0000","last_active":"2024-09-10T04:00:54.441417+0000","last_peered":"2024-09-10T04:00:54.441417+0000","last_clean":"2024-09-10T04:00:54.441417+0000","last_became_active":"2024-09-10T04:00:25.186420+0000","last_became_peered":"2024-09-10T04:00:25.186420+0000","last_unstale":"2024-09-10T04:00:54.441417+0000","last_undegraded":"2024-09-10T04:00:54.441417+0000","last_fullsized":"2024-09-10T04:00:54.441417+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:28.968978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":52,"num_read_kb":38,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442538+0000","last_change":"2024-09-10T04:00:23.068725+0000","last_active":"2024-09-10T04:00:54.442538+0000","last_peered":"2024-09-10T04:00:54.442538+0000","last_clean":"2024-09-10T04:00:54.442538+0000","last_became_active":"2024-09-10T04:00:23.067696+0000","last_became_peered":"2024-09-10T04:00:23.067696+0000","last_unstale":"2024-09-10T04:00:54.442538+0000","last_undegraded":"2024-09-10T04:00:54.442538+0000","last_fullsized":"2024-09-10T04:00:54.442538+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:59:24.195394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640613+0000","last_change":"2024-09-10T04:00:27.108784+0000","last_active":"2024-09-10T04:00:54.640613+0000","last_peered":"2024-09-10T04:00:54.640613+0000","last_clean":"2024-09-10T04:00:54.640613+0000","last_became_active":"2024-09-10T04:00:27.107251+0000","last_became_peered":"2024-09-10T04:00:27.107251+0000","last_unstale":"2024-09-10T04:00:54.640613+0000","last_undegraded":"2024-09-10T04:00:54.640613+0000","last_fullsized":"2024-09-10T04:00:54.640613+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:51.389714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640720+0000","last_change":"2024-09-10T04:00:29.129394+0000","last_active":"2024-09-10T04:00:54.640720+0000","last_peered":"2024-09-10T04:00:54.640720+0000","last_clean":"2024-09-10T04:00:54.640720+0000","last_became_active":"2024-09-10T04:00:29.128736+0000","last_became_peered":"2024-09-10T04:00:29.128736+0000","last_unstale":"2024-09-10T04:00:54.640720+0000","last_undegraded":"2024-09-10T04:00:54.640720+0000","last_fullsized":"2024-09-10T04:00:54.640720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:36.283051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"70'20","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640901+0000","last_change":"2024-09-10T04:00:25.067851+0000","last_active":"2024-09-10T04:00:54.640901+0000","last_peered":"2024-09-10T04:00:54.640901+0000","last_clean":"2024-09-10T04:00:54.640901+0000","last_became_active":"2024-09-10T04:00:25.067533+0000","last_became_peered":"2024-09-10T04:00:25.067533+0000","last_unstale":"2024-09-10T04:00:54.640901+0000","last_undegraded":"2024-09-10T04:00:54.640901+0000","last_fullsized":"2024-09-10T04:00:54.640901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:28.225170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643097+0000","last_change":"2024-09-10T04:00:23.078154+0000","last_active":"2024-09-10T04:00:54.643097+0000","last_peered":"2024-09-10T04:00:54.643097+0000","last_clean":"2024-09-10T04:00:54.643097+0000","last_became_active":"2024-09-10T04:00:23.077698+0000","last_became_peered":"2024-09-10T04:00:23.077698+0000","last_unstale":"2024-09-10T04:00:54.643097+0000","last_undegraded":"2024-09-10T04:00:54.643097+0000","last_fullsized":"2024-09-10T04:00:54.643097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:01.260894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137426+0000","last_change":"2024-09-10T04:00:27.105617+0000","last_active":"2024-09-10T04:01:08.137426+0000","last_peered":"2024-09-10T04:01:08.137426+0000","last_clean":"2024-09-10T04:01:08.137426+0000","last_became_active":"2024-09-10T04:00:27.105357+0000","last_became_peered":"2024-09-10T04:00:27.105357+0000","last_unstale":"2024-09-10T04:01:08.137426+0000","last_undegraded":"2024-09-10T04:01:08.137426+0000","last_fullsized":"2024-09-10T04:01:08.137426+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:14.198231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438493+0000","last_change":"2024-09-10T04:00:29.245685+0000","last_active":"2024-09-10T04:00:54.438493+0000","last_peered":"2024-09-10T04:00:54.438493+0000","last_clean":"2024-09-10T04:00:54.438493+0000","last_became_active":"2024-09-10T04:00:29.245378+0000","last_became_peered":"2024-09-10T04:00:29.245378+0000","last_unstale":"2024-09-10T04:00:54.438493+0000","last_undegraded":"2024-09-10T04:00:54.438493+0000","last_fullsized":"2024-09-10T04:00:54.438493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:30.540449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"70'16","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641969+0000","last_change":"2024-09-10T04:00:25.076304+0000","last_active":"2024-09-10T04:00:54.641969+0000","last_peered":"2024-09-10T04:00:54.641969+0000","last_clean":"2024-09-10T04:00:54.641969+0000","last_became_active":"2024-09-10T04:00:25.075807+0000","last_became_peered":"2024-09-10T04:00:25.075807+0000","last_unstale":"2024-09-10T04:00:54.641969+0000","last_undegraded":"2024-09-10T04:00:54.641969+0000","last_fullsized":"2024-09-10T04:00:54.641969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:11.586626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640261+0000","last_change":"2024-09-10T04:00:23.068931+0000","last_active":"2024-09-10T04:00:54.640261+0000","last_peered":"2024-09-10T04:00:54.640261+0000","last_clean":"2024-09-10T04:00:54.640261+0000","last_became_active":"2024-09-10T04:00:23.068600+0000","last_became_peered":"2024-09-10T04:00:23.068600+0000","last_unstale":"2024-09-10T04:00:54.640261+0000","last_undegraded":"2024-09-10T04:00:54.640261+0000","last_fullsized":"2024-09-10T04:00:54.640261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:32.996986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137452+0000","last_change":"2024-09-10T04:00:27.102534+0000","last_active":"2024-09-10T04:01:08.137452+0000","last_peered":"2024-09-10T04:01:08.137452+0000","last_clean":"2024-09-10T04:01:08.137452+0000","last_became_active":"2024-09-10T04:00:27.102205+0000","last_became_peered":"2024-09-10T04:00:27.102205+0000","last_unstale":"2024-09-10T04:01:08.137452+0000","last_undegraded":"2024-09-10T04:01:08.137452+0000","last_fullsized":"2024-09-10T04:01:08.137452+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:04.611529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639823+0000","last_change":"2024-09-10T04:00:29.139279+0000","last_active":"2024-09-10T04:00:54.639823+0000","last_peered":"2024-09-10T04:00:54.639823+0000","last_clean":"2024-09-10T04:00:54.639823+0000","last_became_active":"2024-09-10T04:00:29.138965+0000","last_became_peered":"2024-09-10T04:00:29.138965+0000","last_unstale":"2024-09-10T04:00:54.639823+0000","last_undegraded":"2024-09-10T04:00:54.639823+0000","last_fullsized":"2024-09-10T04:00:54.639823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:09:43.602935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"70'25","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643628+0000","last_change":"2024-09-10T04:00:25.185194+0000","last_active":"2024-09-10T04:00:54.643628+0000","last_peered":"2024-09-10T04:00:54.643628+0000","last_clean":"2024-09-10T04:00:54.643628+0000","last_became_active":"2024-09-10T04:00:25.184849+0000","last_became_peered":"2024-09-10T04:00:25.184849+0000","last_unstale":"2024-09-10T04:00:54.643628+0000","last_undegraded":"2024-09-10T04:00:54.643628+0000","last_fullsized":"2024-09-10T04:00:54.643628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:03.477188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.644194+0000","last_change":"2024-09-10T04:00:23.074817+0000","last_active":"2024-09-10T04:00:54.644194+0000","last_peered":"2024-09-10T04:00:54.644194+0000","last_clean":"2024-09-10T04:00:54.644194+0000","last_became_active":"2024-09-10T04:00:23.074411+0000","last_became_peered":"2024-09-10T04:00:23.074411+0000","last_unstale":"2024-09-10T04:00:54.644194+0000","last_undegraded":"2024-09-10T04:00:54.644194+0000","last_fullsized":"2024-09-10T04:00:54.644194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:14:45.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638917+0000","last_change":"2024-09-10T04:00:27.106276+0000","last_active":"2024-09-10T04:00:54.638917+0000","last_peered":"2024-09-10T04:00:54.638917+0000","last_clean":"2024-09-10T04:00:54.638917+0000","last_became_active":"2024-09-10T04:00:27.105994+0000","last_became_peered":"2024-09-10T04:00:27.105994+0000","last_unstale":"2024-09-10T04:00:54.638917+0000","last_undegraded":"2024-09-10T04:00:54.638917+0000","last_fullsized":"2024-09-10T04:00:54.638917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:42.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641279+0000","last_change":"2024-09-10T04:00:29.244934+0000","last_active":"2024-09-10T04:00:54.641279+0000","last_peered":"2024-09-10T04:00:54.641279+0000","last_clean":"2024-09-10T04:00:54.641279+0000","last_became_active":"2024-09-10T04:00:29.244547+0000","last_became_peered":"2024-09-10T04:00:29.244547+0000","last_unstale":"2024-09-10T04:00:54.641279+0000","last_undegraded":"2024-09-10T04:00:54.641279+0000","last_fullsized":"2024-09-10T04:00:54.641279+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:10.315022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"70'34","reported_seq":97,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137409+0000","last_change":"2024-09-10T04:00:25.081795+0000","last_active":"2024-09-10T04:01:08.137409+0000","last_peered":"2024-09-10T04:01:08.137409+0000","last_clean":"2024-09-10T04:01:08.137409+0000","last_became_active":"2024-09-10T04:00:25.081482+0000","last_became_peered":"2024-09-10T04:00:25.081482+0000","last_unstale":"2024-09-10T04:01:08.137409+0000","last_undegraded":"2024-09-10T04:01:08.137409+0000","last_fullsized":"2024-09-10T04:01:08.137409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:15:18.615242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":57,"num_read_kb":41,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641028+0000","last_change":"2024-09-10T04:00:23.071920+0000","last_active":"2024-09-10T04:00:54.641028+0000","last_peered":"2024-09-10T04:00:54.641028+0000","last_clean":"2024-09-10T04:00:54.641028+0000","last_became_active":"2024-09-10T04:00:23.071630+0000","last_became_peered":"2024-09-10T04:00:23.071630+0000","last_unstale":"2024-09-10T04:00:54.641028+0000","last_undegraded":"2024-09-10T04:00:54.641028+0000","last_fullsized":"2024-09-10T04:00:54.641028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.971500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639028+0000","last_change":"2024-09-10T04:00:25.067009+0000","last_active":"2024-09-10T04:00:54.639028+0000","last_peered":"2024-09-10T04:00:54.639028+0000","last_clean":"2024-09-10T04:00:54.639028+0000","last_became_active":"2024-09-10T04:00:22.049614+0000","last_became_peered":"2024-09-10T04:00:22.049614+0000","last_unstale":"2024-09-10T04:00:54.639028+0000","last_undegraded":"2024-09-10T04:00:54.639028+0000","last_fullsized":"2024-09-10T04:00:54.639028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:05:23.977305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328407,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640859+0000","last_change":"2024-09-10T04:00:27.101748+0000","last_active":"2024-09-10T04:00:54.640859+0000","last_peered":"2024-09-10T04:00:54.640859+0000","last_clean":"2024-09-10T04:00:54.640859+0000","last_became_active":"2024-09-10T04:00:27.101418+0000","last_became_peered":"2024-09-10T04:00:27.101418+0000","last_unstale":"2024-09-10T04:00:54.640859+0000","last_undegraded":"2024-09-10T04:00:54.640859+0000","last_fullsized":"2024-09-10T04:00:54.640859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:22.743751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442155+0000","last_change":"2024-09-10T04:00:29.131152+0000","last_active":"2024-09-10T04:00:54.442155+0000","last_peered":"2024-09-10T04:00:54.442155+0000","last_clean":"2024-09-10T04:00:54.442155+0000","last_became_active":"2024-09-10T04:00:29.130848+0000","last_became_peered":"2024-09-10T04:00:29.130848+0000","last_unstale":"2024-09-10T04:00:54.442155+0000","last_undegraded":"2024-09-10T04:00:54.442155+0000","last_fullsized":"2024-09-10T04:00:54.442155+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:00.754532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"70'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643560+0000","last_change":"2024-09-10T04:00:25.076122+0000","last_active":"2024-09-10T04:00:54.643560+0000","last_peered":"2024-09-10T04:00:54.643560+0000","last_clean":"2024-09-10T04:00:54.643560+0000","last_became_active":"2024-09-10T04:00:25.075817+0000","last_became_peered":"2024-09-10T04:00:25.075817+0000","last_unstale":"2024-09-10T04:00:54.643560+0000","last_undegraded":"2024-09-10T04:00:54.643560+0000","last_fullsized":"2024-09-10T04:00:54.643560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:39.698560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"62'1","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639687+0000","last_change":"2024-09-10T04:00:23.076891+0000","last_active":"2024-09-10T04:00:54.639687+0000","last_peered":"2024-09-10T04:00:54.639687+0000","last_clean":"2024-09-10T04:00:54.639687+0000","last_became_active":"2024-09-10T04:00:23.076150+0000","last_became_peered":"2024-09-10T04:00:23.076150+0000","last_unstale":"2024-09-10T04:00:54.639687+0000","last_undegraded":"2024-09-10T04:00:54.639687+0000","last_fullsized":"2024-09-10T04:00:54.639687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:44.355280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446733+0000","last_change":"2024-09-10T04:00:25.064819+0000","last_active":"2024-09-10T04:00:54.446733+0000","last_peered":"2024-09-10T04:00:54.446733+0000","last_clean":"2024-09-10T04:00:54.446733+0000","last_became_active":"2024-09-10T04:00:22.051060+0000","last_became_peered":"2024-09-10T04:00:22.051060+0000","last_unstale":"2024-09-10T04:00:54.446733+0000","last_undegraded":"2024-09-10T04:00:54.446733+0000","last_fullsized":"2024-09-10T04:00:54.446733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:04.732524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096656200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447419+0000","last_change":"2024-09-10T04:00:27.100194+0000","last_active":"2024-09-10T04:00:54.447419+0000","last_peered":"2024-09-10T04:00:54.447419+0000","last_clean":"2024-09-10T04:00:54.447419+0000","last_became_active":"2024-09-10T04:00:27.099898+0000","last_became_peered":"2024-09-10T04:00:27.099898+0000","last_unstale":"2024-09-10T04:00:54.447419+0000","last_undegraded":"2024-09-10T04:00:54.447419+0000","last_fullsized":"2024-09-10T04:00:54.447419+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:10.777150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439499+0000","last_change":"2024-09-10T04:00:29.243980+0000","last_active":"2024-09-10T04:00:54.439499+0000","last_peered":"2024-09-10T04:00:54.439499+0000","last_clean":"2024-09-10T04:00:54.439499+0000","last_became_active":"2024-09-10T04:00:29.243678+0000","last_became_peered":"2024-09-10T04:00:29.243678+0000","last_unstale":"2024-09-10T04:00:54.439499+0000","last_undegraded":"2024-09-10T04:00:54.439499+0000","last_fullsized":"2024-09-10T04:00:54.439499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:18:12.247181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642821+0000","last_change":"2024-09-10T04:00:25.077135+0000","last_active":"2024-09-10T04:00:54.642821+0000","last_peered":"2024-09-10T04:00:54.642821+0000","last_clean":"2024-09-10T04:00:54.642821+0000","last_became_active":"2024-09-10T04:00:25.076819+0000","last_became_peered":"2024-09-10T04:00:25.076819+0000","last_unstale":"2024-09-10T04:00:54.642821+0000","last_undegraded":"2024-09-10T04:00:54.642821+0000","last_fullsized":"2024-09-10T04:00:54.642821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:34.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641097+0000","last_change":"2024-09-10T04:00:23.063548+0000","last_active":"2024-09-10T04:00:54.641097+0000","last_peered":"2024-09-10T04:00:54.641097+0000","last_clean":"2024-09-10T04:00:54.641097+0000","last_became_active":"2024-09-10T04:00:23.063273+0000","last_became_peered":"2024-09-10T04:00:23.063273+0000","last_unstale":"2024-09-10T04:00:54.641097+0000","last_undegraded":"2024-09-10T04:00:54.641097+0000","last_fullsized":"2024-09-10T04:00:54.641097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.818456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"70'5","reported_seq":88,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:14.386876+0000","last_change":"2024-09-10T04:00:25.185277+0000","last_active":"2024-09-10T04:01:14.386876+0000","last_peered":"2024-09-10T04:01:14.386876+0000","last_clean":"2024-09-10T04:01:14.386876+0000","last_became_active":"2024-09-10T04:00:22.053896+0000","last_became_peered":"2024-09-10T04:00:22.053896+0000","last_unstale":"2024-09-10T04:01:14.386876+0000","last_undegraded":"2024-09-10T04:01:14.386876+0000","last_fullsized":"2024-09-10T04:01:14.386876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:28.086003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082410000000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":47,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640387+0000","last_change":"2024-09-10T04:00:27.104819+0000","last_active":"2024-09-10T04:00:54.640387+0000","last_peered":"2024-09-10T04:00:54.640387+0000","last_clean":"2024-09-10T04:00:54.640387+0000","last_became_active":"2024-09-10T04:00:27.104551+0000","last_became_peered":"2024-09-10T04:00:27.104551+0000","last_unstale":"2024-09-10T04:00:54.640387+0000","last_undegraded":"2024-09-10T04:00:54.640387+0000","last_fullsized":"2024-09-10T04:00:54.640387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:31.705141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643713+0000","last_change":"2024-09-10T04:00:29.130570+0000","last_active":"2024-09-10T04:00:54.643713+0000","last_peered":"2024-09-10T04:00:54.643713+0000","last_clean":"2024-09-10T04:00:54.643713+0000","last_became_active":"2024-09-10T04:00:29.130154+0000","last_became_peered":"2024-09-10T04:00:29.130154+0000","last_unstale":"2024-09-10T04:00:54.643713+0000","last_undegraded":"2024-09-10T04:00:54.643713+0000","last_fullsized":"2024-09-10T04:00:54.643713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:41:23.706534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137775+0000","last_change":"2024-09-10T04:00:25.185455+0000","last_active":"2024-09-10T04:01:08.137775+0000","last_peered":"2024-09-10T04:01:08.137775+0000","last_clean":"2024-09-10T04:01:08.137775+0000","last_became_active":"2024-09-10T04:00:25.185089+0000","last_became_peered":"2024-09-10T04:00:25.185089+0000","last_unstale":"2024-09-10T04:01:08.137775+0000","last_undegraded":"2024-09-10T04:01:08.137775+0000","last_fullsized":"2024-09-10T04:01:08.137775+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:16:56.457756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441714+0000","last_change":"2024-09-10T04:00:23.066485+0000","last_active":"2024-09-10T04:00:54.441714+0000","last_peered":"2024-09-10T04:00:54.441714+0000","last_clean":"2024-09-10T04:00:54.441714+0000","last_became_active":"2024-09-10T04:00:23.066180+0000","last_became_peered":"2024-09-10T04:00:23.066180+0000","last_unstale":"2024-09-10T04:00:54.441714+0000","last_undegraded":"2024-09-10T04:00:54.441714+0000","last_fullsized":"2024-09-10T04:00:54.441714+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:24:50.627471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":271,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:56.953636+0000","last_change":"2024-09-10T04:00:16.372092+0000","last_active":"2024-09-10T04:00:56.953636+0000","last_peered":"2024-09-10T04:00:56.953636+0000","last_clean":"2024-09-10T04:00:56.953636+0000","last_became_active":"2024-09-10T04:00:16.371625+0000","last_became_peered":"2024-09-10T04:00:16.371625+0000","last_unstale":"2024-09-10T04:00:56.953636+0000","last_undegraded":"2024-09-10T04:00:56.953636+0000","last_fullsized":"2024-09-10T04:00:56.953636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T03:57:54.260744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T03:57:54.260744+0000","last_clean_scrub_stamp":"2024-09-10T03:57:54.260744+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:26:23.746400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439193+0000","last_change":"2024-09-10T04:00:27.107074+0000","last_active":"2024-09-10T04:00:54.439193+0000","last_peered":"2024-09-10T04:00:54.439193+0000","last_clean":"2024-09-10T04:00:54.439193+0000","last_became_active":"2024-09-10T04:00:27.106781+0000","last_became_peered":"2024-09-10T04:00:27.106781+0000","last_unstale":"2024-09-10T04:00:54.439193+0000","last_undegraded":"2024-09-10T04:00:54.439193+0000","last_fullsized":"2024-09-10T04:00:54.439193+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:06.134660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640229+0000","last_change":"2024-09-10T04:00:29.130944+0000","last_active":"2024-09-10T04:00:54.640229+0000","last_peered":"2024-09-10T04:00:54.640229+0000","last_clean":"2024-09-10T04:00:54.640229+0000","last_became_active":"2024-09-10T04:00:29.130619+0000","last_became_peered":"2024-09-10T04:00:29.130619+0000","last_unstale":"2024-09-10T04:00:54.640229+0000","last_undegraded":"2024-09-10T04:00:54.640229+0000","last_fullsized":"2024-09-10T04:00:54.640229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:42.495142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"70'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641571+0000","last_change":"2024-09-10T04:00:25.076171+0000","last_active":"2024-09-10T04:00:54.641571+0000","last_peered":"2024-09-10T04:00:54.641571+0000","last_clean":"2024-09-10T04:00:54.641571+0000","last_became_active":"2024-09-10T04:00:25.075873+0000","last_became_peered":"2024-09-10T04:00:25.075873+0000","last_unstale":"2024-09-10T04:00:54.641571+0000","last_undegraded":"2024-09-10T04:00:54.641571+0000","last_fullsized":"2024-09-10T04:00:54.641571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:28:41.798598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641071+0000","last_change":"2024-09-10T04:00:23.068828+0000","last_active":"2024-09-10T04:00:54.641071+0000","last_peered":"2024-09-10T04:00:54.641071+0000","last_clean":"2024-09-10T04:00:54.641071+0000","last_became_active":"2024-09-10T04:00:23.068520+0000","last_became_peered":"2024-09-10T04:00:23.068520+0000","last_unstale":"2024-09-10T04:00:54.641071+0000","last_undegraded":"2024-09-10T04:00:54.641071+0000","last_fullsized":"2024-09-10T04:00:54.641071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:11.226234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640068+0000","last_change":"2024-09-10T04:00:27.104581+0000","last_active":"2024-09-10T04:00:54.640068+0000","last_peered":"2024-09-10T04:00:54.640068+0000","last_clean":"2024-09-10T04:00:54.640068+0000","last_became_active":"2024-09-10T04:00:27.104285+0000","last_became_peered":"2024-09-10T04:00:27.104285+0000","last_unstale":"2024-09-10T04:00:54.640068+0000","last_undegraded":"2024-09-10T04:00:54.640068+0000","last_fullsized":"2024-09-10T04:00:54.640068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:00:19.736700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441916+0000","last_change":"2024-09-10T04:00:29.245930+0000","last_active":"2024-09-10T04:00:54.441916+0000","last_peered":"2024-09-10T04:00:54.441916+0000","last_clean":"2024-09-10T04:00:54.441916+0000","last_became_active":"2024-09-10T04:00:29.245522+0000","last_became_peered":"2024-09-10T04:00:29.245522+0000","last_unstale":"2024-09-10T04:00:54.441916+0000","last_undegraded":"2024-09-10T04:00:54.441916+0000","last_fullsized":"2024-09-10T04:00:54.441916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:46.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641411+0000","last_change":"2024-09-10T04:00:25.076227+0000","last_active":"2024-09-10T04:00:54.641411+0000","last_peered":"2024-09-10T04:00:54.641411+0000","last_clean":"2024-09-10T04:00:54.641411+0000","last_became_active":"2024-09-10T04:00:25.075892+0000","last_became_peered":"2024-09-10T04:00:25.075892+0000","last_unstale":"2024-09-10T04:00:54.641411+0000","last_undegraded":"2024-09-10T04:00:54.641411+0000","last_fullsized":"2024-09-10T04:00:54.641411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:55.159755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639533+0000","last_change":"2024-09-10T04:00:23.080500+0000","last_active":"2024-09-10T04:00:54.639533+0000","last_peered":"2024-09-10T04:00:54.639533+0000","last_clean":"2024-09-10T04:00:54.639533+0000","last_became_active":"2024-09-10T04:00:23.080148+0000","last_became_peered":"2024-09-10T04:00:23.080148+0000","last_unstale":"2024-09-10T04:00:54.639533+0000","last_undegraded":"2024-09-10T04:00:54.639533+0000","last_fullsized":"2024-09-10T04:00:54.639533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:42.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643815+0000","last_change":"2024-09-10T04:00:27.104337+0000","last_active":"2024-09-10T04:00:54.643815+0000","last_peered":"2024-09-10T04:00:54.643815+0000","last_clean":"2024-09-10T04:00:54.643815+0000","last_became_active":"2024-09-10T04:00:27.104057+0000","last_became_peered":"2024-09-10T04:00:27.104057+0000","last_unstale":"2024-09-10T04:00:54.643815+0000","last_undegraded":"2024-09-10T04:00:54.643815+0000","last_fullsized":"2024-09-10T04:00:54.643815+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:09.179958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447314+0000","last_change":"2024-09-10T04:00:29.129254+0000","last_active":"2024-09-10T04:00:54.447314+0000","last_peered":"2024-09-10T04:00:54.447314+0000","last_clean":"2024-09-10T04:00:54.447314+0000","last_became_active":"2024-09-10T04:00:29.128967+0000","last_became_peered":"2024-09-10T04:00:29.128967+0000","last_unstale":"2024-09-10T04:00:54.447314+0000","last_undegraded":"2024-09-10T04:00:54.447314+0000","last_fullsized":"2024-09-10T04:00:54.447314+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:35:44.415874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640844+0000","last_change":"2024-09-10T04:00:25.069650+0000","last_active":"2024-09-10T04:00:54.640844+0000","last_peered":"2024-09-10T04:00:54.640844+0000","last_clean":"2024-09-10T04:00:54.640844+0000","last_became_active":"2024-09-10T04:00:25.069336+0000","last_became_peered":"2024-09-10T04:00:25.069336+0000","last_unstale":"2024-09-10T04:00:54.640844+0000","last_undegraded":"2024-09-10T04:00:54.640844+0000","last_fullsized":"2024-09-10T04:00:54.640844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:16:58.701301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441983+0000","last_change":"2024-09-10T04:00:23.070935+0000","last_active":"2024-09-10T04:00:54.441983+0000","last_peered":"2024-09-10T04:00:54.441983+0000","last_clean":"2024-09-10T04:00:54.441983+0000","last_became_active":"2024-09-10T04:00:23.070609+0000","last_became_peered":"2024-09-10T04:00:23.070609+0000","last_unstale":"2024-09-10T04:00:54.441983+0000","last_undegraded":"2024-09-10T04:00:54.441983+0000","last_fullsized":"2024-09-10T04:00:54.441983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.826361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137355+0000","last_change":"2024-09-10T04:00:27.100835+0000","last_active":"2024-09-10T04:01:08.137355+0000","last_peered":"2024-09-10T04:01:08.137355+0000","last_clean":"2024-09-10T04:01:08.137355+0000","last_became_active":"2024-09-10T04:00:27.100229+0000","last_became_peered":"2024-09-10T04:00:27.100229+0000","last_unstale":"2024-09-10T04:01:08.137355+0000","last_undegraded":"2024-09-10T04:01:08.137355+0000","last_fullsized":"2024-09-10T04:01:08.137355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:35.318728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"69'1","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641410+0000","last_change":"2024-09-10T04:00:29.138301+0000","last_active":"2024-09-10T04:00:54.641410+0000","last_peered":"2024-09-10T04:00:54.641410+0000","last_clean":"2024-09-10T04:00:54.641410+0000","last_became_active":"2024-09-10T04:00:29.137885+0000","last_became_peered":"2024-09-10T04:00:29.137885+0000","last_unstale":"2024-09-10T04:00:54.641410+0000","last_undegraded":"2024-09-10T04:00:54.641410+0000","last_fullsized":"2024-09-10T04:00:54.641410+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:15.184363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"70'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643502+0000","last_change":"2024-09-10T04:00:25.077713+0000","last_active":"2024-09-10T04:00:54.643502+0000","last_peered":"2024-09-10T04:00:54.643502+0000","last_clean":"2024-09-10T04:00:54.643502+0000","last_became_active":"2024-09-10T04:00:25.076492+0000","last_became_peered":"2024-09-10T04:00:25.076492+0000","last_unstale":"2024-09-10T04:00:54.643502+0000","last_undegraded":"2024-09-10T04:00:54.643502+0000","last_fullsized":"2024-09-10T04:00:54.643502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:27.361193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639680+0000","last_change":"2024-09-10T04:00:23.064694+0000","last_active":"2024-09-10T04:00:54.639680+0000","last_peered":"2024-09-10T04:00:54.639680+0000","last_clean":"2024-09-10T04:00:54.639680+0000","last_became_active":"2024-09-10T04:00:23.063822+0000","last_became_peered":"2024-09-10T04:00:23.063822+0000","last_unstale":"2024-09-10T04:00:54.639680+0000","last_undegraded":"2024-09-10T04:00:54.639680+0000","last_fullsized":"2024-09-10T04:00:54.639680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:13.422935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446806+0000","last_change":"2024-09-10T04:00:27.105417+0000","last_active":"2024-09-10T04:00:54.446806+0000","last_peered":"2024-09-10T04:00:54.446806+0000","last_clean":"2024-09-10T04:00:54.446806+0000","last_became_active":"2024-09-10T04:00:27.105108+0000","last_became_peered":"2024-09-10T04:00:27.105108+0000","last_unstale":"2024-09-10T04:00:54.446806+0000","last_undegraded":"2024-09-10T04:00:54.446806+0000","last_fullsized":"2024-09-10T04:00:54.446806+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:14.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640946+0000","last_change":"2024-09-10T04:00:29.138251+0000","last_active":"2024-09-10T04:00:54.640946+0000","last_peered":"2024-09-10T04:00:54.640946+0000","last_clean":"2024-09-10T04:00:54.640946+0000","last_became_active":"2024-09-10T04:00:29.137899+0000","last_became_peered":"2024-09-10T04:00:29.137899+0000","last_unstale":"2024-09-10T04:00:54.640946+0000","last_undegraded":"2024-09-10T04:00:54.640946+0000","last_fullsized":"2024-09-10T04:00:54.640946+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:02.749682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"70'25","reported_seq":69,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639007+0000","last_change":"2024-09-10T04:00:25.186747+0000","last_active":"2024-09-10T04:00:54.639007+0000","last_peered":"2024-09-10T04:00:54.639007+0000","last_clean":"2024-09-10T04:00:54.639007+0000","last_became_active":"2024-09-10T04:00:25.186345+0000","last_became_peered":"2024-09-10T04:00:25.186345+0000","last_unstale":"2024-09-10T04:00:54.639007+0000","last_undegraded":"2024-09-10T04:00:54.639007+0000","last_fullsized":"2024-09-10T04:00:54.639007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:47.379135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":41,"num_read_kb":29,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439846+0000","last_change":"2024-09-10T04:00:23.066328+0000","last_active":"2024-09-10T04:00:54.439846+0000","last_peered":"2024-09-10T04:00:54.439846+0000","last_clean":"2024-09-10T04:00:54.439846+0000","last_became_active":"2024-09-10T04:00:23.065500+0000","last_became_peered":"2024-09-10T04:00:23.065500+0000","last_unstale":"2024-09-10T04:00:54.439846+0000","last_undegraded":"2024-09-10T04:00:54.439846+0000","last_fullsized":"2024-09-10T04:00:54.439846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:42.282547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446863+0000","last_change":"2024-09-10T04:00:27.101424+0000","last_active":"2024-09-10T04:00:54.446863+0000","last_peered":"2024-09-10T04:00:54.446863+0000","last_clean":"2024-09-10T04:00:54.446863+0000","last_became_active":"2024-09-10T04:00:27.101151+0000","last_became_peered":"2024-09-10T04:00:27.101151+0000","last_unstale":"2024-09-10T04:00:54.446863+0000","last_undegraded":"2024-09-10T04:00:54.446863+0000","last_fullsized":"2024-09-10T04:00:54.446863+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:55:36.488752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438112+0000","last_change":"2024-09-10T04:00:29.136574+0000","last_active":"2024-09-10T04:00:54.438112+0000","last_peered":"2024-09-10T04:00:54.438112+0000","last_clean":"2024-09-10T04:00:54.438112+0000","last_became_active":"2024-09-10T04:00:29.136265+0000","last_became_peered":"2024-09-10T04:00:29.136265+0000","last_unstale":"2024-09-10T04:00:54.438112+0000","last_undegraded":"2024-09-10T04:00:54.438112+0000","last_fullsized":"2024-09-10T04:00:54.438112+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:41.300222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'24","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640618+0000","last_change":"2024-09-10T04:00:25.079652+0000","last_active":"2024-09-10T04:00:54.640618+0000","last_peered":"2024-09-10T04:00:54.640618+0000","last_clean":"2024-09-10T04:00:54.640618+0000","last_became_active":"2024-09-10T04:00:25.079297+0000","last_became_peered":"2024-09-10T04:00:25.079297+0000","last_unstale":"2024-09-10T04:00:54.640618+0000","last_undegraded":"2024-09-10T04:00:54.640618+0000","last_fullsized":"2024-09-10T04:00:54.640618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:44.368142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438649+0000","last_change":"2024-09-10T04:00:23.075836+0000","last_active":"2024-09-10T04:00:54.438649+0000","last_peered":"2024-09-10T04:00:54.438649+0000","last_clean":"2024-09-10T04:00:54.438649+0000","last_became_active":"2024-09-10T04:00:23.075521+0000","last_became_peered":"2024-09-10T04:00:23.075521+0000","last_unstale":"2024-09-10T04:00:54.438649+0000","last_undegraded":"2024-09-10T04:00:54.438649+0000","last_fullsized":"2024-09-10T04:00:54.438649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:27.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446634+0000","last_change":"2024-09-10T04:00:27.098919+0000","last_active":"2024-09-10T04:00:54.446634+0000","last_peered":"2024-09-10T04:00:54.446634+0000","last_clean":"2024-09-10T04:00:54.446634+0000","last_became_active":"2024-09-10T04:00:27.098606+0000","last_became_peered":"2024-09-10T04:00:27.098606+0000","last_unstale":"2024-09-10T04:00:54.446634+0000","last_undegraded":"2024-09-10T04:00:54.446634+0000","last_fullsized":"2024-09-10T04:00:54.446634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:55.170959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441746+0000","last_change":"2024-09-10T04:00:29.129268+0000","last_active":"2024-09-10T04:00:54.441746+0000","last_peered":"2024-09-10T04:00:54.441746+0000","last_clean":"2024-09-10T04:00:54.441746+0000","last_became_active":"2024-09-10T04:00:29.128919+0000","last_became_peered":"2024-09-10T04:00:29.128919+0000","last_unstale":"2024-09-10T04:00:54.441746+0000","last_undegraded":"2024-09-10T04:00:54.441746+0000","last_fullsized":"2024-09-10T04:00:54.441746+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:39:08.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"70'27","reported_seq":75,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638515+0000","last_change":"2024-09-10T04:00:25.186721+0000","last_active":"2024-09-10T04:00:54.638515+0000","last_peered":"2024-09-10T04:00:54.638515+0000","last_clean":"2024-09-10T04:00:54.638515+0000","last_became_active":"2024-09-10T04:00:25.186333+0000","last_became_peered":"2024-09-10T04:00:25.186333+0000","last_unstale":"2024-09-10T04:00:54.638515+0000","last_undegraded":"2024-09-10T04:00:54.638515+0000","last_fullsized":"2024-09-10T04:00:54.638515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:42:28.526051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.440163+0000","last_change":"2024-09-10T04:00:23.067471+0000","last_active":"2024-09-10T04:00:54.440163+0000","last_peered":"2024-09-10T04:00:54.440163+0000","last_clean":"2024-09-10T04:00:54.440163+0000","last_became_active":"2024-09-10T04:00:23.066902+0000","last_became_peered":"2024-09-10T04:00:23.066902+0000","last_unstale":"2024-09-10T04:00:54.440163+0000","last_undegraded":"2024-09-10T04:00:54.440163+0000","last_fullsized":"2024-09-10T04:00:54.440163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:33.001710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137635+0000","last_change":"2024-09-10T04:00:27.103814+0000","last_active":"2024-09-10T04:01:08.137635+0000","last_peered":"2024-09-10T04:01:08.137635+0000","last_clean":"2024-09-10T04:01:08.137635+0000","last_became_active":"2024-09-10T04:00:27.103485+0000","last_became_peered":"2024-09-10T04:00:27.103485+0000","last_unstale":"2024-09-10T04:01:08.137635+0000","last_undegraded":"2024-09-10T04:01:08.137635+0000","last_fullsized":"2024-09-10T04:01:08.137635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.786063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638868+0000","last_change":"2024-09-10T04:00:29.243675+0000","last_active":"2024-09-10T04:00:54.638868+0000","last_peered":"2024-09-10T04:00:54.638868+0000","last_clean":"2024-09-10T04:00:54.638868+0000","last_became_active":"2024-09-10T04:00:29.243338+0000","last_became_peered":"2024-09-10T04:00:29.243338+0000","last_unstale":"2024-09-10T04:00:54.638868+0000","last_undegraded":"2024-09-10T04:00:54.638868+0000","last_fullsized":"2024-09-10T04:00:54.638868+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:09.257379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"62'1","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638790+0000","last_change":"2024-09-10T04:00:23.065261+0000","last_active":"2024-09-10T04:00:54.638790+0000","last_peered":"2024-09-10T04:00:54.638790+0000","last_clean":"2024-09-10T04:00:54.638790+0000","last_became_active":"2024-09-10T04:00:23.064913+0000","last_became_peered":"2024-09-10T04:00:23.064913+0000","last_unstale":"2024-09-10T04:00:54.638790+0000","last_undegraded":"2024-09-10T04:00:54.638790+0000","last_fullsized":"2024-09-10T04:00:54.638790+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'12","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442445+0000","last_change":"2024-09-10T04:00:25.185120+0000","last_active":"2024-09-10T04:00:54.442445+0000","last_peered":"2024-09-10T04:00:54.442445+0000","last_clean":"2024-09-10T04:00:54.442445+0000","last_became_active":"2024-09-10T04:00:25.184828+0000","last_became_peered":"2024-09-10T04:00:25.184828+0000","last_unstale":"2024-09-10T04:00:54.442445+0000","last_undegraded":"2024-09-10T04:00:54.442445+0000","last_fullsized":"2024-09-10T04:00:54.442445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:12.582746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640112+0000","last_change":"2024-09-10T04:00:27.109527+0000","last_active":"2024-09-10T04:00:54.640112+0000","last_peered":"2024-09-10T04:00:54.640112+0000","last_clean":"2024-09-10T04:00:54.640112+0000","last_became_active":"2024-09-10T04:00:27.109206+0000","last_became_peered":"2024-09-10T04:00:27.109206+0000","last_unstale":"2024-09-10T04:00:54.640112+0000","last_undegraded":"2024-09-10T04:00:54.640112+0000","last_fullsized":"2024-09-10T04:00:54.640112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:38.351914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439404+0000","last_change":"2024-09-10T04:00:29.245968+0000","last_active":"2024-09-10T04:00:54.439404+0000","last_peered":"2024-09-10T04:00:54.439404+0000","last_clean":"2024-09-10T04:00:54.439404+0000","last_became_active":"2024-09-10T04:00:29.245244+0000","last_became_peered":"2024-09-10T04:00:29.245244+0000","last_unstale":"2024-09-10T04:00:54.439404+0000","last_undegraded":"2024-09-10T04:00:54.439404+0000","last_fullsized":"2024-09-10T04:00:54.439404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:52.641021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'21","reported_seq":66,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642964+0000","last_change":"2024-09-10T04:00:25.187123+0000","last_active":"2024-09-10T04:00:54.642964+0000","last_peered":"2024-09-10T04:00:54.642964+0000","last_clean":"2024-09-10T04:00:54.642964+0000","last_became_active":"2024-09-10T04:00:25.186823+0000","last_became_peered":"2024-09-10T04:00:25.186823+0000","last_unstale":"2024-09-10T04:00:54.642964+0000","last_undegraded":"2024-09-10T04:00:54.642964+0000","last_fullsized":"2024-09-10T04:00:54.642964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:15.902297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438303+0000","last_change":"2024-09-10T04:00:23.062547+0000","last_active":"2024-09-10T04:00:54.438303+0000","last_peered":"2024-09-10T04:00:54.438303+0000","last_clean":"2024-09-10T04:00:54.438303+0000","last_became_active":"2024-09-10T04:00:23.062127+0000","last_became_peered":"2024-09-10T04:00:23.062127+0000","last_unstale":"2024-09-10T04:00:54.438303+0000","last_undegraded":"2024-09-10T04:00:54.438303+0000","last_fullsized":"2024-09-10T04:00:54.438303+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:38:29.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442357+0000","last_change":"2024-09-10T04:00:27.108710+0000","last_active":"2024-09-10T04:00:54.442357+0000","last_peered":"2024-09-10T04:00:54.442357+0000","last_clean":"2024-09-10T04:00:54.442357+0000","last_became_active":"2024-09-10T04:00:27.108447+0000","last_became_peered":"2024-09-10T04:00:27.108447+0000","last_unstale":"2024-09-10T04:00:54.442357+0000","last_undegraded":"2024-09-10T04:00:54.442357+0000","last_fullsized":"2024-09-10T04:00:54.442357+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:54:27.800551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447001+0000","last_change":"2024-09-10T04:00:29.126663+0000","last_active":"2024-09-10T04:00:54.447001+0000","last_peered":"2024-09-10T04:00:54.447001+0000","last_clean":"2024-09-10T04:00:54.447001+0000","last_became_active":"2024-09-10T04:00:29.126329+0000","last_became_peered":"2024-09-10T04:00:29.126329+0000","last_unstale":"2024-09-10T04:00:54.447001+0000","last_undegraded":"2024-09-10T04:00:54.447001+0000","last_fullsized":"2024-09-10T04:00:54.447001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:09.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"70'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640046+0000","last_change":"2024-09-10T04:00:25.187143+0000","last_active":"2024-09-10T04:00:54.640046+0000","last_peered":"2024-09-10T04:00:54.640046+0000","last_clean":"2024-09-10T04:00:54.640046+0000","last_became_active":"2024-09-10T04:00:25.186746+0000","last_became_peered":"2024-09-10T04:00:25.186746+0000","last_unstale":"2024-09-10T04:00:54.640046+0000","last_undegraded":"2024-09-10T04:00:54.640046+0000","last_fullsized":"2024-09-10T04:00:54.640046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641255+0000","last_change":"2024-09-10T04:00:23.068720+0000","last_active":"2024-09-10T04:00:54.641255+0000","last_peered":"2024-09-10T04:00:54.641255+0000","last_clean":"2024-09-10T04:00:54.641255+0000","last_became_active":"2024-09-10T04:00:23.068420+0000","last_became_peered":"2024-09-10T04:00:23.068420+0000","last_unstale":"2024-09-10T04:00:54.641255+0000","last_undegraded":"2024-09-10T04:00:54.641255+0000","last_fullsized":"2024-09-10T04:00:54.641255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:40.163321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137749+0000","last_change":"2024-09-10T04:00:27.108757+0000","last_active":"2024-09-10T04:01:08.137749+0000","last_peered":"2024-09-10T04:01:08.137749+0000","last_clean":"2024-09-10T04:01:08.137749+0000","last_became_active":"2024-09-10T04:00:27.108425+0000","last_became_peered":"2024-09-10T04:00:27.108425+0000","last_unstale":"2024-09-10T04:01:08.137749+0000","last_undegraded":"2024-09-10T04:01:08.137749+0000","last_fullsized":"2024-09-10T04:01:08.137749+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:03.631143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641249+0000","last_change":"2024-09-10T04:00:29.125783+0000","last_active":"2024-09-10T04:00:54.641249+0000","last_peered":"2024-09-10T04:00:54.641249+0000","last_clean":"2024-09-10T04:00:54.641249+0000","last_became_active":"2024-09-10T04:00:29.125484+0000","last_became_peered":"2024-09-10T04:00:29.125484+0000","last_unstale":"2024-09-10T04:00:54.641249+0000","last_undegraded":"2024-09-10T04:00:54.641249+0000","last_fullsized":"2024-09-10T04:00:54.641249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:47.054742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"70'22","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442159+0000","last_change":"2024-09-10T04:00:25.185118+0000","last_active":"2024-09-10T04:00:54.442159+0000","last_peered":"2024-09-10T04:00:54.442159+0000","last_clean":"2024-09-10T04:00:54.442159+0000","last_became_active":"2024-09-10T04:00:25.184775+0000","last_became_peered":"2024-09-10T04:00:25.184775+0000","last_unstale":"2024-09-10T04:00:54.442159+0000","last_undegraded":"2024-09-10T04:00:54.442159+0000","last_fullsized":"2024-09-10T04:00:54.442159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:11.672340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438806+0000","last_change":"2024-09-10T04:00:23.071007+0000","last_active":"2024-09-10T04:00:54.438806+0000","last_peered":"2024-09-10T04:00:54.438806+0000","last_clean":"2024-09-10T04:00:54.438806+0000","last_became_active":"2024-09-10T04:00:23.070698+0000","last_became_peered":"2024-09-10T04:00:23.070698+0000","last_unstale":"2024-09-10T04:00:54.438806+0000","last_undegraded":"2024-09-10T04:00:54.438806+0000","last_fullsized":"2024-09-10T04:00:54.438806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:13.553023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137636+0000","last_change":"2024-09-10T04:00:27.105320+0000","last_active":"2024-09-10T04:01:08.137636+0000","last_peered":"2024-09-10T04:01:08.137636+0000","last_clean":"2024-09-10T04:01:08.137636+0000","last_became_active":"2024-09-10T04:00:27.105011+0000","last_became_peered":"2024-09-10T04:00:27.105011+0000","last_unstale":"2024-09-10T04:01:08.137636+0000","last_undegraded":"2024-09-10T04:01:08.137636+0000","last_fullsized":"2024-09-10T04:01:08.137636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:54.126227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641791+0000","last_change":"2024-09-10T04:00:29.137672+0000","last_active":"2024-09-10T04:00:54.641791+0000","last_peered":"2024-09-10T04:00:54.641791+0000","last_clean":"2024-09-10T04:00:54.641791+0000","last_became_active":"2024-09-10T04:00:29.137319+0000","last_became_peered":"2024-09-10T04:00:29.137319+0000","last_unstale":"2024-09-10T04:00:54.641791+0000","last_undegraded":"2024-09-10T04:00:54.641791+0000","last_fullsized":"2024-09-10T04:00:54.641791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:44.859748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640796+0000","last_change":"2024-09-10T04:00:25.079477+0000","last_active":"2024-09-10T04:00:54.640796+0000","last_peered":"2024-09-10T04:00:54.640796+0000","last_clean":"2024-09-10T04:00:54.640796+0000","last_became_active":"2024-09-10T04:00:25.079117+0000","last_became_peered":"2024-09-10T04:00:25.079117+0000","last_unstale":"2024-09-10T04:00:54.640796+0000","last_undegraded":"2024-09-10T04:00:54.640796+0000","last_fullsized":"2024-09-10T04:00:54.640796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:51.267018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641100+0000","last_change":"2024-09-10T04:00:23.063490+0000","last_active":"2024-09-10T04:00:54.641100+0000","last_peered":"2024-09-10T04:00:54.641100+0000","last_clean":"2024-09-10T04:00:54.641100+0000","last_became_active":"2024-09-10T04:00:23.063092+0000","last_became_peered":"2024-09-10T04:00:23.063092+0000","last_unstale":"2024-09-10T04:00:54.641100+0000","last_undegraded":"2024-09-10T04:00:54.641100+0000","last_fullsized":"2024-09-10T04:00:54.641100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:10.834112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641817+0000","last_change":"2024-09-10T04:00:27.106419+0000","last_active":"2024-09-10T04:00:54.641817+0000","last_peered":"2024-09-10T04:00:54.641817+0000","last_clean":"2024-09-10T04:00:54.641817+0000","last_became_active":"2024-09-10T04:00:27.106125+0000","last_became_peered":"2024-09-10T04:00:27.106125+0000","last_unstale":"2024-09-10T04:00:54.641817+0000","last_undegraded":"2024-09-10T04:00:54.641817+0000","last_fullsized":"2024-09-10T04:00:54.641817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:25:42.827308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441358+0000","last_change":"2024-09-10T04:00:29.136190+0000","last_active":"2024-09-10T04:00:54.441358+0000","last_peered":"2024-09-10T04:00:54.441358+0000","last_clean":"2024-09-10T04:00:54.441358+0000","last_became_active":"2024-09-10T04:00:29.135889+0000","last_became_peered":"2024-09-10T04:00:29.135889+0000","last_unstale":"2024-09-10T04:00:54.441358+0000","last_undegraded":"2024-09-10T04:00:54.441358+0000","last_fullsized":"2024-09-10T04:00:54.441358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:59.654169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642198+0000","last_change":"2024-09-10T04:00:25.075537+0000","last_active":"2024-09-10T04:00:54.642198+0000","last_peered":"2024-09-10T04:00:54.642198+0000","last_clean":"2024-09-10T04:00:54.642198+0000","last_became_active":"2024-09-10T04:00:25.075230+0000","last_became_peered":"2024-09-10T04:00:25.075230+0000","last_unstale":"2024-09-10T04:00:54.642198+0000","last_undegraded":"2024-09-10T04:00:54.642198+0000","last_fullsized":"2024-09-10T04:00:54.642198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:13.774853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439703+0000","last_change":"2024-09-10T04:00:23.067084+0000","last_active":"2024-09-10T04:00:54.439703+0000","last_peered":"2024-09-10T04:00:54.439703+0000","last_clean":"2024-09-10T04:00:54.439703+0000","last_became_active":"2024-09-10T04:00:23.066772+0000","last_became_peered":"2024-09-10T04:00:23.066772+0000","last_unstale":"2024-09-10T04:00:54.439703+0000","last_undegraded":"2024-09-10T04:00:54.439703+0000","last_fullsized":"2024-09-10T04:00:54.439703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:59.182575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441839+0000","last_change":"2024-09-10T04:00:27.104865+0000","last_active":"2024-09-10T04:00:54.441839+0000","last_peered":"2024-09-10T04:00:54.441839+0000","last_clean":"2024-09-10T04:00:54.441839+0000","last_became_active":"2024-09-10T04:00:27.104594+0000","last_became_peered":"2024-09-10T04:00:27.104594+0000","last_unstale":"2024-09-10T04:00:54.441839+0000","last_undegraded":"2024-09-10T04:00:54.441839+0000","last_fullsized":"2024-09-10T04:00:54.441839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:48:51.304827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639413+0000","last_change":"2024-09-10T04:00:29.139566+0000","last_active":"2024-09-10T04:00:54.639413+0000","last_peered":"2024-09-10T04:00:54.639413+0000","last_clean":"2024-09-10T04:00:54.639413+0000","last_became_active":"2024-09-10T04:00:29.139228+0000","last_became_peered":"2024-09-10T04:00:29.139228+0000","last_unstale":"2024-09-10T04:00:54.639413+0000","last_undegraded":"2024-09-10T04:00:54.639413+0000","last_fullsized":"2024-09-10T04:00:54.639413+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:28.697922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"70'18","reported_seq":58,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441495+0000","last_change":"2024-09-10T04:00:25.185154+0000","last_active":"2024-09-10T04:00:54.441495+0000","last_peered":"2024-09-10T04:00:54.441495+0000","last_clean":"2024-09-10T04:00:54.441495+0000","last_became_active":"2024-09-10T04:00:25.184863+0000","last_became_peered":"2024-09-10T04:00:25.184863+0000","last_unstale":"2024-09-10T04:00:54.441495+0000","last_undegraded":"2024-09-10T04:00:54.441495+0000","last_fullsized":"2024-09-10T04:00:54.441495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:16.314765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640203+0000","last_change":"2024-09-10T04:00:23.075927+0000","last_active":"2024-09-10T04:00:54.640203+0000","last_peered":"2024-09-10T04:00:54.640203+0000","last_clean":"2024-09-10T04:00:54.640203+0000","last_became_active":"2024-09-10T04:00:23.075534+0000","last_became_peered":"2024-09-10T04:00:23.075534+0000","last_unstale":"2024-09-10T04:00:54.640203+0000","last_undegraded":"2024-09-10T04:00:54.640203+0000","last_fullsized":"2024-09-10T04:00:54.640203+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:48.618323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.437867+0000","last_change":"2024-09-10T04:00:27.104739+0000","last_active":"2024-09-10T04:00:54.437867+0000","last_peered":"2024-09-10T04:00:54.437867+0000","last_clean":"2024-09-10T04:00:54.437867+0000","last_became_active":"2024-09-10T04:00:27.104205+0000","last_became_peered":"2024-09-10T04:00:27.104205+0000","last_unstale":"2024-09-10T04:00:54.437867+0000","last_undegraded":"2024-09-10T04:00:54.437867+0000","last_fullsized":"2024-09-10T04:00:54.437867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:04.153702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441922+0000","last_change":"2024-09-10T04:00:29.243928+0000","last_active":"2024-09-10T04:00:54.441922+0000","last_peered":"2024-09-10T04:00:54.441922+0000","last_clean":"2024-09-10T04:00:54.441922+0000","last_became_active":"2024-09-10T04:00:29.243545+0000","last_became_peered":"2024-09-10T04:00:29.243545+0000","last_unstale":"2024-09-10T04:00:54.441922+0000","last_undegraded":"2024-09-10T04:00:54.441922+0000","last_fullsized":"2024-09-10T04:00:54.441922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:09:14.153640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441368+0000","last_change":"2024-09-10T04:00:25.079602+0000","last_active":"2024-09-10T04:00:54.441368+0000","last_peered":"2024-09-10T04:00:54.441368+0000","last_clean":"2024-09-10T04:00:54.441368+0000","last_became_active":"2024-09-10T04:00:25.079300+0000","last_became_peered":"2024-09-10T04:00:25.079300+0000","last_unstale":"2024-09-10T04:00:54.441368+0000","last_undegraded":"2024-09-10T04:00:54.441368+0000","last_fullsized":"2024-09-10T04:00:54.441368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:02:45.390334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641041+0000","last_change":"2024-09-10T04:00:23.069048+0000","last_active":"2024-09-10T04:00:54.641041+0000","last_peered":"2024-09-10T04:00:54.641041+0000","last_clean":"2024-09-10T04:00:54.641041+0000","last_became_active":"2024-09-10T04:00:23.068792+0000","last_became_peered":"2024-09-10T04:00:23.068792+0000","last_unstale":"2024-09-10T04:00:54.641041+0000","last_undegraded":"2024-09-10T04:00:54.641041+0000","last_fullsized":"2024-09-10T04:00:54.641041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:57.535680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638674+0000","last_change":"2024-09-10T04:00:27.107361+0000","last_active":"2024-09-10T04:00:54.638674+0000","last_peered":"2024-09-10T04:00:54.638674+0000","last_clean":"2024-09-10T04:00:54.638674+0000","last_became_active":"2024-09-10T04:00:27.107078+0000","last_became_peered":"2024-09-10T04:00:27.107078+0000","last_unstale":"2024-09-10T04:00:54.638674+0000","last_undegraded":"2024-09-10T04:00:54.638674+0000","last_fullsized":"2024-09-10T04:00:54.638674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:03:44.442007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438121+0000","last_change":"2024-09-10T04:00:29.128837+0000","last_active":"2024-09-10T04:00:54.438121+0000","last_peered":"2024-09-10T04:00:54.438121+0000","last_clean":"2024-09-10T04:00:54.438121+0000","last_became_active":"2024-09-10T04:00:29.128517+0000","last_became_peered":"2024-09-10T04:00:29.128517+0000","last_unstale":"2024-09-10T04:00:54.438121+0000","last_undegraded":"2024-09-10T04:00:54.438121+0000","last_fullsized":"2024-09-10T04:00:54.438121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:42.488602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642606+0000","last_change":"2024-09-10T04:00:29.132840+0000","last_active":"2024-09-10T04:00:54.642606+0000","last_peered":"2024-09-10T04:00:54.642606+0000","last_clean":"2024-09-10T04:00:54.642606+0000","last_became_active":"2024-09-10T04:00:29.132467+0000","last_became_peered":"2024-09-10T04:00:29.132467+0000","last_unstale":"2024-09-10T04:00:54.642606+0000","last_undegraded":"2024-09-10T04:00:54.642606+0000","last_fullsized":"2024-09-10T04:00:54.642606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:00:01.943463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441645+0000","last_change":"2024-09-10T04:00:23.077521+0000","last_active":"2024-09-10T04:00:54.441645+0000","last_peered":"2024-09-10T04:00:54.441645+0000","last_clean":"2024-09-10T04:00:54.441645+0000","last_became_active":"2024-09-10T04:00:23.077243+0000","last_became_peered":"2024-09-10T04:00:23.077243+0000","last_unstale":"2024-09-10T04:00:54.441645+0000","last_undegraded":"2024-09-10T04:00:54.441645+0000","last_fullsized":"2024-09-10T04:00:54.441645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:14:01.098627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639800+0000","last_change":"2024-09-10T04:00:25.074605+0000","last_active":"2024-09-10T04:00:54.639800+0000","last_peered":"2024-09-10T04:00:54.639800+0000","last_clean":"2024-09-10T04:00:54.639800+0000","last_became_active":"2024-09-10T04:00:25.074281+0000","last_became_peered":"2024-09-10T04:00:25.074281+0000","last_unstale":"2024-09-10T04:00:54.639800+0000","last_undegraded":"2024-09-10T04:00:54.639800+0000","last_fullsized":"2024-09-10T04:00:54.639800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:19.219164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641930+0000","last_change":"2024-09-10T04:00:27.100686+0000","last_active":"2024-09-10T04:00:54.641930+0000","last_peered":"2024-09-10T04:00:54.641930+0000","last_clean":"2024-09-10T04:00:54.641930+0000","last_became_active":"2024-09-10T04:00:27.100382+0000","last_became_peered":"2024-09-10T04:00:27.100382+0000","last_unstale":"2024-09-10T04:00:54.641930+0000","last_undegraded":"2024-09-10T04:00:54.641930+0000","last_fullsized":"2024-09-10T04:00:54.641930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:34:46.630185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":986,"num_read_kb":711,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":586,"ondisk_log_size":586,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":42,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201296,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28376,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712680,"statfs":{"total":95990841344,"available":95961784320,"internally_reserved":0,"allocated":1384448,"data_stored":916542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462933,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28364,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712692,"statfs":{"total":95990841344,"available":95961796608,"internally_reserved":0,"allocated":1372160,"data_stored":916886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93709108,"statfs":{"total":95990841344,"available":95958126592,"internally_reserved":0,"allocated":778240,"data_stored":325884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986207,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713244,"statfs":{"total":95990841344,"available":95962361856,"internally_reserved":0,"allocated":802816,"data_stored":325707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149734,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93709080,"statfs":{"total":95990841344,"available":95958097920,"internally_reserved":0,"allocated":806912,"data_stored":325980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713272,"statfs":{"total":95990841344,"available":95962390528,"internally_reserved":0,"allocated":774144,"data_stored":326243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713204,"statfs":{"total":95990841344,"available":95962320896,"internally_reserved":0,"allocated":843776,"data_stored":327880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708512,"statfs":{"total":95990841344,"available":95957516288,"internally_reserved":0,"allocated":1392640,"data_stored":916687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T04:01:19.361 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T04:01:19.670 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:19.754 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:19.462Z caller=cluster.go:708 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000159379s 2024-09-10T04:01:19.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[25393]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[25393]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[25393]: Deploying daemon grafana.a on smithi050 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[25393]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[25393]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[30767]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[30767]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[30767]: Deploying daemon grafana.a on smithi050 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[30767]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:19.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:19 smithi038 ceph-mon[30767]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:20.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:19 smithi050 ceph-mon[26687]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T04:01:20.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:19 smithi050 ceph-mon[26687]: from='mon.? -' entity='mon.' cmd=[{"prefix": "orch prometheus get-credentials"}]: dispatch 2024-09-10T04:01:20.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:19 smithi050 ceph-mon[26687]: Deploying daemon grafana.a on smithi050 2024-09-10T04:01:20.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:19 smithi050 ceph-mon[26687]: from='client.24623 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:20.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:19 smithi050 ceph-mon[26687]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:21.012 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:01:21.013 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-10T04:01:21.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:20 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:21.255 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:20 smithi038 ceph-mon[25393]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:20 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:21.255 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:20 smithi038 ceph-mon[30767]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:20 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:21.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:20 smithi050 ceph-mon[26687]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:21.680 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-10T04:01:20.459144+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":231,"num_object_clones":0,"num_object_copies":693,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":231,"num_whiteouts":0,"num_read":1213,"num_read_kb":1035,"num_write":807,"num_write_kb":3239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":805,"ondisk_log_size":805,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":236656,"kb_used_data":7964,"kb_used_omap":12,"kb_used_meta":228531,"kb_avail":749691792,"statfs":{"total":767926730752,"available":767684395008,"internally_reserved":0,"allocated":8155136,"data_stored":4381809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":234016307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003136"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640953+0000","last_change":"2024-09-10T04:00:23.072686+0000","last_active":"2024-09-10T04:00:54.640953+0000","last_peered":"2024-09-10T04:00:54.640953+0000","last_clean":"2024-09-10T04:00:54.640953+0000","last_became_active":"2024-09-10T04:00:23.072412+0000","last_became_peered":"2024-09-10T04:00:23.072412+0000","last_unstale":"2024-09-10T04:00:54.640953+0000","last_undegraded":"2024-09-10T04:00:54.640953+0000","last_fullsized":"2024-09-10T04:00:54.640953+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:01:35.269691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"69'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640446+0000","last_change":"2024-09-10T04:00:25.076327+0000","last_active":"2024-09-10T04:00:54.640446+0000","last_peered":"2024-09-10T04:00:54.640446+0000","last_clean":"2024-09-10T04:00:54.640446+0000","last_became_active":"2024-09-10T04:00:25.076009+0000","last_became_peered":"2024-09-10T04:00:25.076009+0000","last_unstale":"2024-09-10T04:00:54.640446+0000","last_undegraded":"2024-09-10T04:00:54.640446+0000","last_fullsized":"2024-09-10T04:00:54.640446+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:30:02.311326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641396+0000","last_change":"2024-09-10T04:00:27.105165+0000","last_active":"2024-09-10T04:00:54.641396+0000","last_peered":"2024-09-10T04:00:54.641396+0000","last_clean":"2024-09-10T04:00:54.641396+0000","last_became_active":"2024-09-10T04:00:27.104904+0000","last_became_peered":"2024-09-10T04:00:27.104904+0000","last_unstale":"2024-09-10T04:00:54.641396+0000","last_undegraded":"2024-09-10T04:00:54.641396+0000","last_fullsized":"2024-09-10T04:00:54.641396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:19:21.256230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640612+0000","last_change":"2024-09-10T04:00:29.134903+0000","last_active":"2024-09-10T04:00:54.640612+0000","last_peered":"2024-09-10T04:00:54.640612+0000","last_clean":"2024-09-10T04:00:54.640612+0000","last_became_active":"2024-09-10T04:00:29.134519+0000","last_became_peered":"2024-09-10T04:00:29.134519+0000","last_unstale":"2024-09-10T04:00:54.640612+0000","last_undegraded":"2024-09-10T04:00:54.640612+0000","last_fullsized":"2024-09-10T04:00:54.640612+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:56:12.176160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441613+0000","last_change":"2024-09-10T04:00:29.246054+0000","last_active":"2024-09-10T04:00:54.441613+0000","last_peered":"2024-09-10T04:00:54.441613+0000","last_clean":"2024-09-10T04:00:54.441613+0000","last_became_active":"2024-09-10T04:00:29.245687+0000","last_became_peered":"2024-09-10T04:00:29.245687+0000","last_unstale":"2024-09-10T04:00:54.441613+0000","last_undegraded":"2024-09-10T04:00:54.441613+0000","last_fullsized":"2024-09-10T04:00:54.441613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:59:28.145580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442454+0000","last_change":"2024-09-10T04:00:23.078699+0000","last_active":"2024-09-10T04:00:54.442454+0000","last_peered":"2024-09-10T04:00:54.442454+0000","last_clean":"2024-09-10T04:00:54.442454+0000","last_became_active":"2024-09-10T04:00:23.078396+0000","last_became_peered":"2024-09-10T04:00:23.078396+0000","last_unstale":"2024-09-10T04:00:54.442454+0000","last_undegraded":"2024-09-10T04:00:54.442454+0000","last_fullsized":"2024-09-10T04:00:54.442454+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:39:31.182539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"70'39","reported_seq":99,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441650+0000","last_change":"2024-09-10T04:00:25.073606+0000","last_active":"2024-09-10T04:00:54.441650+0000","last_peered":"2024-09-10T04:00:54.441650+0000","last_clean":"2024-09-10T04:00:54.441650+0000","last_became_active":"2024-09-10T04:00:25.073287+0000","last_became_peered":"2024-09-10T04:00:25.073287+0000","last_unstale":"2024-09-10T04:00:54.441650+0000","last_undegraded":"2024-09-10T04:00:54.441650+0000","last_fullsized":"2024-09-10T04:00:54.441650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:11:56.652958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":62,"num_read_kb":49,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640358+0000","last_change":"2024-09-10T04:00:27.102876+0000","last_active":"2024-09-10T04:00:54.640358+0000","last_peered":"2024-09-10T04:00:54.640358+0000","last_clean":"2024-09-10T04:00:54.640358+0000","last_became_active":"2024-09-10T04:00:27.102604+0000","last_became_peered":"2024-09-10T04:00:27.102604+0000","last_unstale":"2024-09-10T04:00:54.640358+0000","last_undegraded":"2024-09-10T04:00:54.640358+0000","last_fullsized":"2024-09-10T04:00:54.640358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:59.508066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640171+0000","last_change":"2024-09-10T04:00:23.060679+0000","last_active":"2024-09-10T04:00:54.640171+0000","last_peered":"2024-09-10T04:00:54.640171+0000","last_clean":"2024-09-10T04:00:54.640171+0000","last_became_active":"2024-09-10T04:00:23.060219+0000","last_became_peered":"2024-09-10T04:00:23.060219+0000","last_unstale":"2024-09-10T04:00:54.640171+0000","last_undegraded":"2024-09-10T04:00:54.640171+0000","last_fullsized":"2024-09-10T04:00:54.640171+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:13:20.124081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"70'21","reported_seq":66,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640742+0000","last_change":"2024-09-10T04:00:25.070774+0000","last_active":"2024-09-10T04:00:54.640742+0000","last_peered":"2024-09-10T04:00:54.640742+0000","last_clean":"2024-09-10T04:00:54.640742+0000","last_became_active":"2024-09-10T04:00:25.070422+0000","last_became_peered":"2024-09-10T04:00:25.070422+0000","last_unstale":"2024-09-10T04:00:54.640742+0000","last_undegraded":"2024-09-10T04:00:54.640742+0000","last_fullsized":"2024-09-10T04:00:54.640742+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:07:11.044787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638765+0000","last_change":"2024-09-10T04:00:27.103057+0000","last_active":"2024-09-10T04:00:54.638765+0000","last_peered":"2024-09-10T04:00:54.638765+0000","last_clean":"2024-09-10T04:00:54.638765+0000","last_became_active":"2024-09-10T04:00:27.102772+0000","last_became_peered":"2024-09-10T04:00:27.102772+0000","last_unstale":"2024-09-10T04:00:54.638765+0000","last_undegraded":"2024-09-10T04:00:54.638765+0000","last_fullsized":"2024-09-10T04:00:54.638765+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:56:04.069894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639588+0000","last_change":"2024-09-10T04:00:29.134358+0000","last_active":"2024-09-10T04:00:54.639588+0000","last_peered":"2024-09-10T04:00:54.639588+0000","last_clean":"2024-09-10T04:00:54.639588+0000","last_became_active":"2024-09-10T04:00:29.133979+0000","last_became_peered":"2024-09-10T04:00:29.133979+0000","last_unstale":"2024-09-10T04:00:54.639588+0000","last_undegraded":"2024-09-10T04:00:54.639588+0000","last_fullsized":"2024-09-10T04:00:54.639588+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:31:35.904688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639509+0000","last_change":"2024-09-10T04:00:23.076834+0000","last_active":"2024-09-10T04:00:54.639509+0000","last_peered":"2024-09-10T04:00:54.639509+0000","last_clean":"2024-09-10T04:00:54.639509+0000","last_became_active":"2024-09-10T04:00:23.076398+0000","last_became_peered":"2024-09-10T04:00:23.076398+0000","last_unstale":"2024-09-10T04:00:54.639509+0000","last_undegraded":"2024-09-10T04:00:54.639509+0000","last_fullsized":"2024-09-10T04:00:54.639509+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:32:39.606592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"70'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640670+0000","last_change":"2024-09-10T04:00:25.079520+0000","last_active":"2024-09-10T04:00:54.640670+0000","last_peered":"2024-09-10T04:00:54.640670+0000","last_clean":"2024-09-10T04:00:54.640670+0000","last_became_active":"2024-09-10T04:00:25.079006+0000","last_became_peered":"2024-09-10T04:00:25.079006+0000","last_unstale":"2024-09-10T04:00:54.640670+0000","last_undegraded":"2024-09-10T04:00:54.640670+0000","last_fullsized":"2024-09-10T04:00:54.640670+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:52:59.277084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":15,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438211+0000","last_change":"2024-09-10T04:00:27.105721+0000","last_active":"2024-09-10T04:00:54.438211+0000","last_peered":"2024-09-10T04:00:54.438211+0000","last_clean":"2024-09-10T04:00:54.438211+0000","last_became_active":"2024-09-10T04:00:27.105431+0000","last_became_peered":"2024-09-10T04:00:27.105431+0000","last_unstale":"2024-09-10T04:00:54.438211+0000","last_undegraded":"2024-09-10T04:00:54.438211+0000","last_fullsized":"2024-09-10T04:00:54.438211+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:40:44.569653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638664+0000","last_change":"2024-09-10T04:00:29.129999+0000","last_active":"2024-09-10T04:00:54.638664+0000","last_peered":"2024-09-10T04:00:54.638664+0000","last_clean":"2024-09-10T04:00:54.638664+0000","last_became_active":"2024-09-10T04:00:29.129706+0000","last_became_peered":"2024-09-10T04:00:29.129706+0000","last_unstale":"2024-09-10T04:00:54.638664+0000","last_undegraded":"2024-09-10T04:00:54.638664+0000","last_fullsized":"2024-09-10T04:00:54.638664+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:51:37.456263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641438+0000","last_change":"2024-09-10T04:00:29.245498+0000","last_active":"2024-09-10T04:00:54.641438+0000","last_peered":"2024-09-10T04:00:54.641438+0000","last_clean":"2024-09-10T04:00:54.641438+0000","last_became_active":"2024-09-10T04:00:29.245161+0000","last_became_peered":"2024-09-10T04:00:29.245161+0000","last_unstale":"2024-09-10T04:00:54.641438+0000","last_undegraded":"2024-09-10T04:00:54.641438+0000","last_fullsized":"2024-09-10T04:00:54.641438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:59:15.406497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.644046+0000","last_change":"2024-09-10T04:00:23.070438+0000","last_active":"2024-09-10T04:00:54.644046+0000","last_peered":"2024-09-10T04:00:54.644046+0000","last_clean":"2024-09-10T04:00:54.644046+0000","last_became_active":"2024-09-10T04:00:23.070085+0000","last_became_peered":"2024-09-10T04:00:23.070085+0000","last_unstale":"2024-09-10T04:00:54.644046+0000","last_undegraded":"2024-09-10T04:00:54.644046+0000","last_fullsized":"2024-09-10T04:00:54.644046+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:32:38.781922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"70'15","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447238+0000","last_change":"2024-09-10T04:00:25.077613+0000","last_active":"2024-09-10T04:00:54.447238+0000","last_peered":"2024-09-10T04:00:54.447238+0000","last_clean":"2024-09-10T04:00:54.447238+0000","last_became_active":"2024-09-10T04:00:25.077282+0000","last_became_peered":"2024-09-10T04:00:25.077282+0000","last_unstale":"2024-09-10T04:00:54.447238+0000","last_undegraded":"2024-09-10T04:00:54.447238+0000","last_fullsized":"2024-09-10T04:00:54.447238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:37:44.850925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642848+0000","last_change":"2024-09-10T04:00:27.106867+0000","last_active":"2024-09-10T04:00:54.642848+0000","last_peered":"2024-09-10T04:00:54.642848+0000","last_clean":"2024-09-10T04:00:54.642848+0000","last_became_active":"2024-09-10T04:00:27.106419+0000","last_became_peered":"2024-09-10T04:00:27.106419+0000","last_unstale":"2024-09-10T04:00:54.642848+0000","last_undegraded":"2024-09-10T04:00:54.642848+0000","last_fullsized":"2024-09-10T04:00:54.642848+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:33.749714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442334+0000","last_change":"2024-09-10T04:00:29.245925+0000","last_active":"2024-09-10T04:00:54.442334+0000","last_peered":"2024-09-10T04:00:54.442334+0000","last_clean":"2024-09-10T04:00:54.442334+0000","last_became_active":"2024-09-10T04:00:29.245504+0000","last_became_peered":"2024-09-10T04:00:29.245504+0000","last_unstale":"2024-09-10T04:00:54.442334+0000","last_undegraded":"2024-09-10T04:00:54.442334+0000","last_fullsized":"2024-09-10T04:00:54.442334+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:13.293950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640900+0000","last_change":"2024-09-10T04:00:23.080555+0000","last_active":"2024-09-10T04:00:54.640900+0000","last_peered":"2024-09-10T04:00:54.640900+0000","last_clean":"2024-09-10T04:00:54.640900+0000","last_became_active":"2024-09-10T04:00:23.079901+0000","last_became_peered":"2024-09-10T04:00:23.079901+0000","last_unstale":"2024-09-10T04:00:54.640900+0000","last_undegraded":"2024-09-10T04:00:54.640900+0000","last_fullsized":"2024-09-10T04:00:54.640900+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:57:02.017897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"70'18","reported_seq":61,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442124+0000","last_change":"2024-09-10T04:00:25.076449+0000","last_active":"2024-09-10T04:00:54.442124+0000","last_peered":"2024-09-10T04:00:54.442124+0000","last_clean":"2024-09-10T04:00:54.442124+0000","last_became_active":"2024-09-10T04:00:25.076116+0000","last_became_peered":"2024-09-10T04:00:25.076116+0000","last_unstale":"2024-09-10T04:00:54.442124+0000","last_undegraded":"2024-09-10T04:00:54.442124+0000","last_fullsized":"2024-09-10T04:00:54.442124+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:58:29.508435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":24,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640746+0000","last_change":"2024-09-10T04:00:27.106013+0000","last_active":"2024-09-10T04:00:54.640746+0000","last_peered":"2024-09-10T04:00:54.640746+0000","last_clean":"2024-09-10T04:00:54.640746+0000","last_became_active":"2024-09-10T04:00:27.105741+0000","last_became_peered":"2024-09-10T04:00:27.105741+0000","last_unstale":"2024-09-10T04:00:54.640746+0000","last_undegraded":"2024-09-10T04:00:54.640746+0000","last_fullsized":"2024-09-10T04:00:54.640746+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:20:21.840639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"69'1","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438962+0000","last_change":"2024-09-10T04:00:29.136695+0000","last_active":"2024-09-10T04:00:54.438962+0000","last_peered":"2024-09-10T04:00:54.438962+0000","last_clean":"2024-09-10T04:00:54.438962+0000","last_became_active":"2024-09-10T04:00:29.136386+0000","last_became_peered":"2024-09-10T04:00:29.136386+0000","last_unstale":"2024-09-10T04:00:54.438962+0000","last_undegraded":"2024-09-10T04:00:54.438962+0000","last_fullsized":"2024-09-10T04:00:54.438962+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:50:38.894928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"62'1","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641580+0000","last_change":"2024-09-10T04:00:23.074974+0000","last_active":"2024-09-10T04:00:54.641580+0000","last_peered":"2024-09-10T04:00:54.641580+0000","last_clean":"2024-09-10T04:00:54.641580+0000","last_became_active":"2024-09-10T04:00:23.074610+0000","last_became_peered":"2024-09-10T04:00:23.074610+0000","last_unstale":"2024-09-10T04:00:54.641580+0000","last_undegraded":"2024-09-10T04:00:54.641580+0000","last_fullsized":"2024-09-10T04:00:54.641580+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:01:08.194672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"70'16","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642143+0000","last_change":"2024-09-10T04:00:25.186355+0000","last_active":"2024-09-10T04:00:54.642143+0000","last_peered":"2024-09-10T04:00:54.642143+0000","last_clean":"2024-09-10T04:00:54.642143+0000","last_became_active":"2024-09-10T04:00:25.185910+0000","last_became_peered":"2024-09-10T04:00:25.185910+0000","last_unstale":"2024-09-10T04:00:54.642143+0000","last_undegraded":"2024-09-10T04:00:54.642143+0000","last_fullsized":"2024-09-10T04:00:54.642143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:07:43.441887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"69'8","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137852+0000","last_change":"2024-09-10T04:00:27.104212+0000","last_active":"2024-09-10T04:01:13.137852+0000","last_peered":"2024-09-10T04:01:13.137852+0000","last_clean":"2024-09-10T04:01:13.137852+0000","last_became_active":"2024-09-10T04:00:27.103921+0000","last_became_peered":"2024-09-10T04:00:27.103921+0000","last_unstale":"2024-09-10T04:01:13.137852+0000","last_undegraded":"2024-09-10T04:01:13.137852+0000","last_fullsized":"2024-09-10T04:01:13.137852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:41:38.029153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"70'15","reported_seq":51,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643396+0000","last_change":"2024-09-10T04:00:25.080380+0000","last_active":"2024-09-10T04:00:54.643396+0000","last_peered":"2024-09-10T04:00:54.643396+0000","last_clean":"2024-09-10T04:00:54.643396+0000","last_became_active":"2024-09-10T04:00:25.080076+0000","last_became_peered":"2024-09-10T04:00:25.080076+0000","last_unstale":"2024-09-10T04:00:54.643396+0000","last_undegraded":"2024-09-10T04:00:54.643396+0000","last_fullsized":"2024-09-10T04:00:54.643396+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:53:48.997357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441736+0000","last_change":"2024-09-10T04:00:23.076719+0000","last_active":"2024-09-10T04:00:54.441736+0000","last_peered":"2024-09-10T04:00:54.441736+0000","last_clean":"2024-09-10T04:00:54.441736+0000","last_became_active":"2024-09-10T04:00:23.076027+0000","last_became_peered":"2024-09-10T04:00:23.076027+0000","last_unstale":"2024-09-10T04:00:54.441736+0000","last_undegraded":"2024-09-10T04:00:54.441736+0000","last_fullsized":"2024-09-10T04:00:54.441736+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:08:04.555840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"69'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137914+0000","last_change":"2024-09-10T04:00:27.102303+0000","last_active":"2024-09-10T04:01:13.137914+0000","last_peered":"2024-09-10T04:01:13.137914+0000","last_clean":"2024-09-10T04:01:13.137914+0000","last_became_active":"2024-09-10T04:00:27.101831+0000","last_became_peered":"2024-09-10T04:00:27.101831+0000","last_unstale":"2024-09-10T04:01:13.137914+0000","last_undegraded":"2024-09-10T04:01:13.137914+0000","last_fullsized":"2024-09-10T04:01:13.137914+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:43:20.576793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639212+0000","last_change":"2024-09-10T04:00:29.134284+0000","last_active":"2024-09-10T04:00:54.639212+0000","last_peered":"2024-09-10T04:00:54.639212+0000","last_clean":"2024-09-10T04:00:54.639212+0000","last_became_active":"2024-09-10T04:00:29.133969+0000","last_became_peered":"2024-09-10T04:00:29.133969+0000","last_unstale":"2024-09-10T04:00:54.639212+0000","last_undegraded":"2024-09-10T04:00:54.639212+0000","last_fullsized":"2024-09-10T04:00:54.639212+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:29:30.203485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"70'30","reported_seq":82,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441417+0000","last_change":"2024-09-10T04:00:25.186762+0000","last_active":"2024-09-10T04:00:54.441417+0000","last_peered":"2024-09-10T04:00:54.441417+0000","last_clean":"2024-09-10T04:00:54.441417+0000","last_became_active":"2024-09-10T04:00:25.186420+0000","last_became_peered":"2024-09-10T04:00:25.186420+0000","last_unstale":"2024-09-10T04:00:54.441417+0000","last_undegraded":"2024-09-10T04:00:54.441417+0000","last_fullsized":"2024-09-10T04:00:54.441417+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:11:28.968978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":52,"num_read_kb":38,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442538+0000","last_change":"2024-09-10T04:00:23.068725+0000","last_active":"2024-09-10T04:00:54.442538+0000","last_peered":"2024-09-10T04:00:54.442538+0000","last_clean":"2024-09-10T04:00:54.442538+0000","last_became_active":"2024-09-10T04:00:23.067696+0000","last_became_peered":"2024-09-10T04:00:23.067696+0000","last_unstale":"2024-09-10T04:00:54.442538+0000","last_undegraded":"2024-09-10T04:00:54.442538+0000","last_fullsized":"2024-09-10T04:00:54.442538+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:59:24.195394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640613+0000","last_change":"2024-09-10T04:00:27.108784+0000","last_active":"2024-09-10T04:00:54.640613+0000","last_peered":"2024-09-10T04:00:54.640613+0000","last_clean":"2024-09-10T04:00:54.640613+0000","last_became_active":"2024-09-10T04:00:27.107251+0000","last_became_peered":"2024-09-10T04:00:27.107251+0000","last_unstale":"2024-09-10T04:00:54.640613+0000","last_undegraded":"2024-09-10T04:00:54.640613+0000","last_fullsized":"2024-09-10T04:00:54.640613+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:07:51.389714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640720+0000","last_change":"2024-09-10T04:00:29.129394+0000","last_active":"2024-09-10T04:00:54.640720+0000","last_peered":"2024-09-10T04:00:54.640720+0000","last_clean":"2024-09-10T04:00:54.640720+0000","last_became_active":"2024-09-10T04:00:29.128736+0000","last_became_peered":"2024-09-10T04:00:29.128736+0000","last_unstale":"2024-09-10T04:00:54.640720+0000","last_undegraded":"2024-09-10T04:00:54.640720+0000","last_fullsized":"2024-09-10T04:00:54.640720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:03:36.283051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"70'20","reported_seq":59,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640901+0000","last_change":"2024-09-10T04:00:25.067851+0000","last_active":"2024-09-10T04:00:54.640901+0000","last_peered":"2024-09-10T04:00:54.640901+0000","last_clean":"2024-09-10T04:00:54.640901+0000","last_became_active":"2024-09-10T04:00:25.067533+0000","last_became_peered":"2024-09-10T04:00:25.067533+0000","last_unstale":"2024-09-10T04:00:54.640901+0000","last_undegraded":"2024-09-10T04:00:54.640901+0000","last_fullsized":"2024-09-10T04:00:54.640901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:45:28.225170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"62'1","reported_seq":31,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643097+0000","last_change":"2024-09-10T04:00:23.078154+0000","last_active":"2024-09-10T04:00:54.643097+0000","last_peered":"2024-09-10T04:00:54.643097+0000","last_clean":"2024-09-10T04:00:54.643097+0000","last_became_active":"2024-09-10T04:00:23.077698+0000","last_became_peered":"2024-09-10T04:00:23.077698+0000","last_unstale":"2024-09-10T04:00:54.643097+0000","last_undegraded":"2024-09-10T04:00:54.643097+0000","last_fullsized":"2024-09-10T04:00:54.643097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:05:01.260894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"69'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137548+0000","last_change":"2024-09-10T04:00:27.105617+0000","last_active":"2024-09-10T04:01:13.137548+0000","last_peered":"2024-09-10T04:01:13.137548+0000","last_clean":"2024-09-10T04:01:13.137548+0000","last_became_active":"2024-09-10T04:00:27.105357+0000","last_became_peered":"2024-09-10T04:00:27.105357+0000","last_unstale":"2024-09-10T04:01:13.137548+0000","last_undegraded":"2024-09-10T04:01:13.137548+0000","last_fullsized":"2024-09-10T04:01:13.137548+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:13:14.198231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438493+0000","last_change":"2024-09-10T04:00:29.245685+0000","last_active":"2024-09-10T04:00:54.438493+0000","last_peered":"2024-09-10T04:00:54.438493+0000","last_clean":"2024-09-10T04:00:54.438493+0000","last_became_active":"2024-09-10T04:00:29.245378+0000","last_became_peered":"2024-09-10T04:00:29.245378+0000","last_unstale":"2024-09-10T04:00:54.438493+0000","last_undegraded":"2024-09-10T04:00:54.438493+0000","last_fullsized":"2024-09-10T04:00:54.438493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:59:30.540449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"70'16","reported_seq":53,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641969+0000","last_change":"2024-09-10T04:00:25.076304+0000","last_active":"2024-09-10T04:00:54.641969+0000","last_peered":"2024-09-10T04:00:54.641969+0000","last_clean":"2024-09-10T04:00:54.641969+0000","last_became_active":"2024-09-10T04:00:25.075807+0000","last_became_peered":"2024-09-10T04:00:25.075807+0000","last_unstale":"2024-09-10T04:00:54.641969+0000","last_undegraded":"2024-09-10T04:00:54.641969+0000","last_fullsized":"2024-09-10T04:00:54.641969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:12:11.586626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640261+0000","last_change":"2024-09-10T04:00:23.068931+0000","last_active":"2024-09-10T04:00:54.640261+0000","last_peered":"2024-09-10T04:00:54.640261+0000","last_clean":"2024-09-10T04:00:54.640261+0000","last_became_active":"2024-09-10T04:00:23.068600+0000","last_became_peered":"2024-09-10T04:00:23.068600+0000","last_unstale":"2024-09-10T04:00:54.640261+0000","last_undegraded":"2024-09-10T04:00:54.640261+0000","last_fullsized":"2024-09-10T04:00:54.640261+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:20:32.996986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"69'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137509+0000","last_change":"2024-09-10T04:00:27.102534+0000","last_active":"2024-09-10T04:01:13.137509+0000","last_peered":"2024-09-10T04:01:13.137509+0000","last_clean":"2024-09-10T04:01:13.137509+0000","last_became_active":"2024-09-10T04:00:27.102205+0000","last_became_peered":"2024-09-10T04:00:27.102205+0000","last_unstale":"2024-09-10T04:01:13.137509+0000","last_undegraded":"2024-09-10T04:01:13.137509+0000","last_fullsized":"2024-09-10T04:01:13.137509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:07:04.611529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639823+0000","last_change":"2024-09-10T04:00:29.139279+0000","last_active":"2024-09-10T04:00:54.639823+0000","last_peered":"2024-09-10T04:00:54.639823+0000","last_clean":"2024-09-10T04:00:54.639823+0000","last_became_active":"2024-09-10T04:00:29.138965+0000","last_became_peered":"2024-09-10T04:00:29.138965+0000","last_unstale":"2024-09-10T04:00:54.639823+0000","last_undegraded":"2024-09-10T04:00:54.639823+0000","last_fullsized":"2024-09-10T04:00:54.639823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:09:43.602935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"70'25","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643628+0000","last_change":"2024-09-10T04:00:25.185194+0000","last_active":"2024-09-10T04:00:54.643628+0000","last_peered":"2024-09-10T04:00:54.643628+0000","last_clean":"2024-09-10T04:00:54.643628+0000","last_became_active":"2024-09-10T04:00:25.184849+0000","last_became_peered":"2024-09-10T04:00:25.184849+0000","last_unstale":"2024-09-10T04:00:54.643628+0000","last_undegraded":"2024-09-10T04:00:54.643628+0000","last_fullsized":"2024-09-10T04:00:54.643628+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:03.477188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.644194+0000","last_change":"2024-09-10T04:00:23.074817+0000","last_active":"2024-09-10T04:00:54.644194+0000","last_peered":"2024-09-10T04:00:54.644194+0000","last_clean":"2024-09-10T04:00:54.644194+0000","last_became_active":"2024-09-10T04:00:23.074411+0000","last_became_peered":"2024-09-10T04:00:23.074411+0000","last_unstale":"2024-09-10T04:00:54.644194+0000","last_undegraded":"2024-09-10T04:00:54.644194+0000","last_fullsized":"2024-09-10T04:00:54.644194+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:14:45.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638917+0000","last_change":"2024-09-10T04:00:27.106276+0000","last_active":"2024-09-10T04:00:54.638917+0000","last_peered":"2024-09-10T04:00:54.638917+0000","last_clean":"2024-09-10T04:00:54.638917+0000","last_became_active":"2024-09-10T04:00:27.105994+0000","last_became_peered":"2024-09-10T04:00:27.105994+0000","last_unstale":"2024-09-10T04:00:54.638917+0000","last_undegraded":"2024-09-10T04:00:54.638917+0000","last_fullsized":"2024-09-10T04:00:54.638917+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:26:42.023942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641279+0000","last_change":"2024-09-10T04:00:29.244934+0000","last_active":"2024-09-10T04:00:54.641279+0000","last_peered":"2024-09-10T04:00:54.641279+0000","last_clean":"2024-09-10T04:00:54.641279+0000","last_became_active":"2024-09-10T04:00:29.244547+0000","last_became_peered":"2024-09-10T04:00:29.244547+0000","last_unstale":"2024-09-10T04:00:54.641279+0000","last_undegraded":"2024-09-10T04:00:54.641279+0000","last_fullsized":"2024-09-10T04:00:54.641279+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:43:10.315022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"70'34","reported_seq":97,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137409+0000","last_change":"2024-09-10T04:00:25.081795+0000","last_active":"2024-09-10T04:01:08.137409+0000","last_peered":"2024-09-10T04:01:08.137409+0000","last_clean":"2024-09-10T04:01:08.137409+0000","last_became_active":"2024-09-10T04:00:25.081482+0000","last_became_peered":"2024-09-10T04:00:25.081482+0000","last_unstale":"2024-09-10T04:01:08.137409+0000","last_undegraded":"2024-09-10T04:01:08.137409+0000","last_fullsized":"2024-09-10T04:01:08.137409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:15:18.615242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":57,"num_read_kb":41,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641028+0000","last_change":"2024-09-10T04:00:23.071920+0000","last_active":"2024-09-10T04:00:54.641028+0000","last_peered":"2024-09-10T04:00:54.641028+0000","last_clean":"2024-09-10T04:00:54.641028+0000","last_became_active":"2024-09-10T04:00:23.071630+0000","last_became_peered":"2024-09-10T04:00:23.071630+0000","last_unstale":"2024-09-10T04:00:54.641028+0000","last_undegraded":"2024-09-10T04:00:54.641028+0000","last_fullsized":"2024-09-10T04:00:54.641028+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:57:45.971500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"64'2","reported_seq":40,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639028+0000","last_change":"2024-09-10T04:00:25.067009+0000","last_active":"2024-09-10T04:00:54.639028+0000","last_peered":"2024-09-10T04:00:54.639028+0000","last_clean":"2024-09-10T04:00:54.639028+0000","last_became_active":"2024-09-10T04:00:22.049614+0000","last_became_peered":"2024-09-10T04:00:22.049614+0000","last_unstale":"2024-09-10T04:00:54.639028+0000","last_undegraded":"2024-09-10T04:00:54.639028+0000","last_fullsized":"2024-09-10T04:00:54.639028+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:05:23.977305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001328407,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640859+0000","last_change":"2024-09-10T04:00:27.101748+0000","last_active":"2024-09-10T04:00:54.640859+0000","last_peered":"2024-09-10T04:00:54.640859+0000","last_clean":"2024-09-10T04:00:54.640859+0000","last_became_active":"2024-09-10T04:00:27.101418+0000","last_became_peered":"2024-09-10T04:00:27.101418+0000","last_unstale":"2024-09-10T04:00:54.640859+0000","last_undegraded":"2024-09-10T04:00:54.640859+0000","last_fullsized":"2024-09-10T04:00:54.640859+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:46:22.743751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442155+0000","last_change":"2024-09-10T04:00:29.131152+0000","last_active":"2024-09-10T04:00:54.442155+0000","last_peered":"2024-09-10T04:00:54.442155+0000","last_clean":"2024-09-10T04:00:54.442155+0000","last_became_active":"2024-09-10T04:00:29.130848+0000","last_became_peered":"2024-09-10T04:00:29.130848+0000","last_unstale":"2024-09-10T04:00:54.442155+0000","last_undegraded":"2024-09-10T04:00:54.442155+0000","last_fullsized":"2024-09-10T04:00:54.442155+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:55:00.754532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"70'19","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643560+0000","last_change":"2024-09-10T04:00:25.076122+0000","last_active":"2024-09-10T04:00:54.643560+0000","last_peered":"2024-09-10T04:00:54.643560+0000","last_clean":"2024-09-10T04:00:54.643560+0000","last_became_active":"2024-09-10T04:00:25.075817+0000","last_became_peered":"2024-09-10T04:00:25.075817+0000","last_unstale":"2024-09-10T04:00:54.643560+0000","last_undegraded":"2024-09-10T04:00:54.643560+0000","last_fullsized":"2024-09-10T04:00:54.643560+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:47:39.698560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":39,"num_read_kb":27,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"62'1","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639687+0000","last_change":"2024-09-10T04:00:23.076891+0000","last_active":"2024-09-10T04:00:54.639687+0000","last_peered":"2024-09-10T04:00:54.639687+0000","last_clean":"2024-09-10T04:00:54.639687+0000","last_became_active":"2024-09-10T04:00:23.076150+0000","last_became_peered":"2024-09-10T04:00:23.076150+0000","last_unstale":"2024-09-10T04:00:54.639687+0000","last_undegraded":"2024-09-10T04:00:54.639687+0000","last_fullsized":"2024-09-10T04:00:54.639687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:46:44.355280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"62'1","reported_seq":39,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446733+0000","last_change":"2024-09-10T04:00:25.064819+0000","last_active":"2024-09-10T04:00:54.446733+0000","last_peered":"2024-09-10T04:00:54.446733+0000","last_clean":"2024-09-10T04:00:54.446733+0000","last_became_active":"2024-09-10T04:00:22.051060+0000","last_became_peered":"2024-09-10T04:00:22.051060+0000","last_unstale":"2024-09-10T04:00:54.446733+0000","last_undegraded":"2024-09-10T04:00:54.446733+0000","last_fullsized":"2024-09-10T04:00:54.446733+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:12:04.732524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096656200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447419+0000","last_change":"2024-09-10T04:00:27.100194+0000","last_active":"2024-09-10T04:00:54.447419+0000","last_peered":"2024-09-10T04:00:54.447419+0000","last_clean":"2024-09-10T04:00:54.447419+0000","last_became_active":"2024-09-10T04:00:27.099898+0000","last_became_peered":"2024-09-10T04:00:27.099898+0000","last_unstale":"2024-09-10T04:00:54.447419+0000","last_undegraded":"2024-09-10T04:00:54.447419+0000","last_fullsized":"2024-09-10T04:00:54.447419+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:38:10.777150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439499+0000","last_change":"2024-09-10T04:00:29.243980+0000","last_active":"2024-09-10T04:00:54.439499+0000","last_peered":"2024-09-10T04:00:54.439499+0000","last_clean":"2024-09-10T04:00:54.439499+0000","last_became_active":"2024-09-10T04:00:29.243678+0000","last_became_peered":"2024-09-10T04:00:29.243678+0000","last_unstale":"2024-09-10T04:00:54.439499+0000","last_undegraded":"2024-09-10T04:00:54.439499+0000","last_fullsized":"2024-09-10T04:00:54.439499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:18:12.247181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"70'12","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642821+0000","last_change":"2024-09-10T04:00:25.077135+0000","last_active":"2024-09-10T04:00:54.642821+0000","last_peered":"2024-09-10T04:00:54.642821+0000","last_clean":"2024-09-10T04:00:54.642821+0000","last_became_active":"2024-09-10T04:00:25.076819+0000","last_became_peered":"2024-09-10T04:00:25.076819+0000","last_unstale":"2024-09-10T04:00:54.642821+0000","last_undegraded":"2024-09-10T04:00:54.642821+0000","last_fullsized":"2024-09-10T04:00:54.642821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:17:34.680247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641097+0000","last_change":"2024-09-10T04:00:23.063548+0000","last_active":"2024-09-10T04:00:54.641097+0000","last_peered":"2024-09-10T04:00:54.641097+0000","last_clean":"2024-09-10T04:00:54.641097+0000","last_became_active":"2024-09-10T04:00:23.063273+0000","last_became_peered":"2024-09-10T04:00:23.063273+0000","last_unstale":"2024-09-10T04:00:54.641097+0000","last_undegraded":"2024-09-10T04:00:54.641097+0000","last_fullsized":"2024-09-10T04:00:54.641097+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:06.818456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"70'5","reported_seq":88,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:14.386876+0000","last_change":"2024-09-10T04:00:25.185277+0000","last_active":"2024-09-10T04:01:14.386876+0000","last_peered":"2024-09-10T04:01:14.386876+0000","last_clean":"2024-09-10T04:01:14.386876+0000","last_became_active":"2024-09-10T04:00:22.053896+0000","last_became_peered":"2024-09-10T04:00:22.053896+0000","last_unstale":"2024-09-10T04:01:14.386876+0000","last_undegraded":"2024-09-10T04:01:14.386876+0000","last_fullsized":"2024-09-10T04:01:14.386876+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:20.998842+0000","last_clean_scrub_stamp":"2024-09-10T04:00:20.998842+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:28.086003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082410000000000003,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":47,"num_read_kb":42,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640387+0000","last_change":"2024-09-10T04:00:27.104819+0000","last_active":"2024-09-10T04:00:54.640387+0000","last_peered":"2024-09-10T04:00:54.640387+0000","last_clean":"2024-09-10T04:00:54.640387+0000","last_became_active":"2024-09-10T04:00:27.104551+0000","last_became_peered":"2024-09-10T04:00:27.104551+0000","last_unstale":"2024-09-10T04:00:54.640387+0000","last_undegraded":"2024-09-10T04:00:54.640387+0000","last_fullsized":"2024-09-10T04:00:54.640387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:14:31.705141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643713+0000","last_change":"2024-09-10T04:00:29.130570+0000","last_active":"2024-09-10T04:00:54.643713+0000","last_peered":"2024-09-10T04:00:54.643713+0000","last_clean":"2024-09-10T04:00:54.643713+0000","last_became_active":"2024-09-10T04:00:29.130154+0000","last_became_peered":"2024-09-10T04:00:29.130154+0000","last_unstale":"2024-09-10T04:00:54.643713+0000","last_undegraded":"2024-09-10T04:00:54.643713+0000","last_fullsized":"2024-09-10T04:00:54.643713+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:41:23.706534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"70'16","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137858+0000","last_change":"2024-09-10T04:00:25.185455+0000","last_active":"2024-09-10T04:01:13.137858+0000","last_peered":"2024-09-10T04:01:13.137858+0000","last_clean":"2024-09-10T04:01:13.137858+0000","last_became_active":"2024-09-10T04:00:25.185089+0000","last_became_peered":"2024-09-10T04:00:25.185089+0000","last_unstale":"2024-09-10T04:01:13.137858+0000","last_undegraded":"2024-09-10T04:01:13.137858+0000","last_fullsized":"2024-09-10T04:01:13.137858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:16:56.457756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441714+0000","last_change":"2024-09-10T04:00:23.066485+0000","last_active":"2024-09-10T04:00:54.441714+0000","last_peered":"2024-09-10T04:00:54.441714+0000","last_clean":"2024-09-10T04:00:54.441714+0000","last_became_active":"2024-09-10T04:00:23.066180+0000","last_became_peered":"2024-09-10T04:00:23.066180+0000","last_unstale":"2024-09-10T04:00:54.441714+0000","last_undegraded":"2024-09-10T04:00:54.441714+0000","last_fullsized":"2024-09-10T04:00:54.441714+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:24:50.627471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":271,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:56.953636+0000","last_change":"2024-09-10T04:00:16.372092+0000","last_active":"2024-09-10T04:00:56.953636+0000","last_peered":"2024-09-10T04:00:56.953636+0000","last_clean":"2024-09-10T04:00:56.953636+0000","last_became_active":"2024-09-10T04:00:16.371625+0000","last_became_peered":"2024-09-10T04:00:16.371625+0000","last_unstale":"2024-09-10T04:00:56.953636+0000","last_undegraded":"2024-09-10T04:00:56.953636+0000","last_fullsized":"2024-09-10T04:00:56.953636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T03:57:54.260744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T03:57:54.260744+0000","last_clean_scrub_stamp":"2024-09-10T03:57:54.260744+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:26:23.746400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439193+0000","last_change":"2024-09-10T04:00:27.107074+0000","last_active":"2024-09-10T04:00:54.439193+0000","last_peered":"2024-09-10T04:00:54.439193+0000","last_clean":"2024-09-10T04:00:54.439193+0000","last_became_active":"2024-09-10T04:00:27.106781+0000","last_became_peered":"2024-09-10T04:00:27.106781+0000","last_unstale":"2024-09-10T04:00:54.439193+0000","last_undegraded":"2024-09-10T04:00:54.439193+0000","last_fullsized":"2024-09-10T04:00:54.439193+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:40:06.134660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640229+0000","last_change":"2024-09-10T04:00:29.130944+0000","last_active":"2024-09-10T04:00:54.640229+0000","last_peered":"2024-09-10T04:00:54.640229+0000","last_clean":"2024-09-10T04:00:54.640229+0000","last_became_active":"2024-09-10T04:00:29.130619+0000","last_became_peered":"2024-09-10T04:00:29.130619+0000","last_unstale":"2024-09-10T04:00:54.640229+0000","last_undegraded":"2024-09-10T04:00:54.640229+0000","last_fullsized":"2024-09-10T04:00:54.640229+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:02:42.495142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"70'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641571+0000","last_change":"2024-09-10T04:00:25.076171+0000","last_active":"2024-09-10T04:00:54.641571+0000","last_peered":"2024-09-10T04:00:54.641571+0000","last_clean":"2024-09-10T04:00:54.641571+0000","last_became_active":"2024-09-10T04:00:25.075873+0000","last_became_peered":"2024-09-10T04:00:25.075873+0000","last_unstale":"2024-09-10T04:00:54.641571+0000","last_undegraded":"2024-09-10T04:00:54.641571+0000","last_fullsized":"2024-09-10T04:00:54.641571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:28:41.798598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641071+0000","last_change":"2024-09-10T04:00:23.068828+0000","last_active":"2024-09-10T04:00:54.641071+0000","last_peered":"2024-09-10T04:00:54.641071+0000","last_clean":"2024-09-10T04:00:54.641071+0000","last_became_active":"2024-09-10T04:00:23.068520+0000","last_became_peered":"2024-09-10T04:00:23.068520+0000","last_unstale":"2024-09-10T04:00:54.641071+0000","last_undegraded":"2024-09-10T04:00:54.641071+0000","last_fullsized":"2024-09-10T04:00:54.641071+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:11.226234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640068+0000","last_change":"2024-09-10T04:00:27.104581+0000","last_active":"2024-09-10T04:00:54.640068+0000","last_peered":"2024-09-10T04:00:54.640068+0000","last_clean":"2024-09-10T04:00:54.640068+0000","last_became_active":"2024-09-10T04:00:27.104285+0000","last_became_peered":"2024-09-10T04:00:27.104285+0000","last_unstale":"2024-09-10T04:00:54.640068+0000","last_undegraded":"2024-09-10T04:00:54.640068+0000","last_fullsized":"2024-09-10T04:00:54.640068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:00:19.736700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441916+0000","last_change":"2024-09-10T04:00:29.245930+0000","last_active":"2024-09-10T04:00:54.441916+0000","last_peered":"2024-09-10T04:00:54.441916+0000","last_clean":"2024-09-10T04:00:54.441916+0000","last_became_active":"2024-09-10T04:00:29.245522+0000","last_became_peered":"2024-09-10T04:00:29.245522+0000","last_unstale":"2024-09-10T04:00:54.441916+0000","last_undegraded":"2024-09-10T04:00:54.441916+0000","last_fullsized":"2024-09-10T04:00:54.441916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:15:46.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"70'17","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641411+0000","last_change":"2024-09-10T04:00:25.076227+0000","last_active":"2024-09-10T04:00:54.641411+0000","last_peered":"2024-09-10T04:00:54.641411+0000","last_clean":"2024-09-10T04:00:54.641411+0000","last_became_active":"2024-09-10T04:00:25.075892+0000","last_became_peered":"2024-09-10T04:00:25.075892+0000","last_unstale":"2024-09-10T04:00:54.641411+0000","last_undegraded":"2024-09-10T04:00:54.641411+0000","last_fullsized":"2024-09-10T04:00:54.641411+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:04:55.159755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639533+0000","last_change":"2024-09-10T04:00:23.080500+0000","last_active":"2024-09-10T04:00:54.639533+0000","last_peered":"2024-09-10T04:00:54.639533+0000","last_clean":"2024-09-10T04:00:54.639533+0000","last_became_active":"2024-09-10T04:00:23.080148+0000","last_became_peered":"2024-09-10T04:00:23.080148+0000","last_unstale":"2024-09-10T04:00:54.639533+0000","last_undegraded":"2024-09-10T04:00:54.639533+0000","last_fullsized":"2024-09-10T04:00:54.639533+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:04:42.856905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643815+0000","last_change":"2024-09-10T04:00:27.104337+0000","last_active":"2024-09-10T04:00:54.643815+0000","last_peered":"2024-09-10T04:00:54.643815+0000","last_clean":"2024-09-10T04:00:54.643815+0000","last_became_active":"2024-09-10T04:00:27.104057+0000","last_became_peered":"2024-09-10T04:00:27.104057+0000","last_unstale":"2024-09-10T04:00:54.643815+0000","last_undegraded":"2024-09-10T04:00:54.643815+0000","last_fullsized":"2024-09-10T04:00:54.643815+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:35:09.179958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447314+0000","last_change":"2024-09-10T04:00:29.129254+0000","last_active":"2024-09-10T04:00:54.447314+0000","last_peered":"2024-09-10T04:00:54.447314+0000","last_clean":"2024-09-10T04:00:54.447314+0000","last_became_active":"2024-09-10T04:00:29.128967+0000","last_became_peered":"2024-09-10T04:00:29.128967+0000","last_unstale":"2024-09-10T04:00:54.447314+0000","last_undegraded":"2024-09-10T04:00:54.447314+0000","last_fullsized":"2024-09-10T04:00:54.447314+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:35:44.415874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"70'10","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640844+0000","last_change":"2024-09-10T04:00:25.069650+0000","last_active":"2024-09-10T04:00:54.640844+0000","last_peered":"2024-09-10T04:00:54.640844+0000","last_clean":"2024-09-10T04:00:54.640844+0000","last_became_active":"2024-09-10T04:00:25.069336+0000","last_became_peered":"2024-09-10T04:00:25.069336+0000","last_unstale":"2024-09-10T04:00:54.640844+0000","last_undegraded":"2024-09-10T04:00:54.640844+0000","last_fullsized":"2024-09-10T04:00:54.640844+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:16:58.701301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441983+0000","last_change":"2024-09-10T04:00:23.070935+0000","last_active":"2024-09-10T04:00:54.441983+0000","last_peered":"2024-09-10T04:00:54.441983+0000","last_clean":"2024-09-10T04:00:54.441983+0000","last_became_active":"2024-09-10T04:00:23.070609+0000","last_became_peered":"2024-09-10T04:00:23.070609+0000","last_unstale":"2024-09-10T04:00:54.441983+0000","last_undegraded":"2024-09-10T04:00:54.441983+0000","last_fullsized":"2024-09-10T04:00:54.441983+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:46:26.826361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"69'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137512+0000","last_change":"2024-09-10T04:00:27.100835+0000","last_active":"2024-09-10T04:01:13.137512+0000","last_peered":"2024-09-10T04:01:13.137512+0000","last_clean":"2024-09-10T04:01:13.137512+0000","last_became_active":"2024-09-10T04:00:27.100229+0000","last_became_peered":"2024-09-10T04:00:27.100229+0000","last_unstale":"2024-09-10T04:01:13.137512+0000","last_undegraded":"2024-09-10T04:01:13.137512+0000","last_fullsized":"2024-09-10T04:01:13.137512+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:12:35.318728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"69'1","reported_seq":19,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641410+0000","last_change":"2024-09-10T04:00:29.138301+0000","last_active":"2024-09-10T04:00:54.641410+0000","last_peered":"2024-09-10T04:00:54.641410+0000","last_clean":"2024-09-10T04:00:54.641410+0000","last_became_active":"2024-09-10T04:00:29.137885+0000","last_became_peered":"2024-09-10T04:00:29.137885+0000","last_unstale":"2024-09-10T04:00:54.641410+0000","last_undegraded":"2024-09-10T04:00:54.641410+0000","last_fullsized":"2024-09-10T04:00:54.641410+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:59:15.184363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"70'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.643502+0000","last_change":"2024-09-10T04:00:25.077713+0000","last_active":"2024-09-10T04:00:54.643502+0000","last_peered":"2024-09-10T04:00:54.643502+0000","last_clean":"2024-09-10T04:00:54.643502+0000","last_became_active":"2024-09-10T04:00:25.076492+0000","last_became_peered":"2024-09-10T04:00:25.076492+0000","last_unstale":"2024-09-10T04:00:54.643502+0000","last_undegraded":"2024-09-10T04:00:54.643502+0000","last_fullsized":"2024-09-10T04:00:54.643502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:56:27.361193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639680+0000","last_change":"2024-09-10T04:00:23.064694+0000","last_active":"2024-09-10T04:00:54.639680+0000","last_peered":"2024-09-10T04:00:54.639680+0000","last_clean":"2024-09-10T04:00:54.639680+0000","last_became_active":"2024-09-10T04:00:23.063822+0000","last_became_peered":"2024-09-10T04:00:23.063822+0000","last_unstale":"2024-09-10T04:00:54.639680+0000","last_undegraded":"2024-09-10T04:00:54.639680+0000","last_fullsized":"2024-09-10T04:00:54.639680+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:13.422935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446806+0000","last_change":"2024-09-10T04:00:27.105417+0000","last_active":"2024-09-10T04:00:54.446806+0000","last_peered":"2024-09-10T04:00:54.446806+0000","last_clean":"2024-09-10T04:00:54.446806+0000","last_became_active":"2024-09-10T04:00:27.105108+0000","last_became_peered":"2024-09-10T04:00:27.105108+0000","last_unstale":"2024-09-10T04:00:54.446806+0000","last_undegraded":"2024-09-10T04:00:54.446806+0000","last_fullsized":"2024-09-10T04:00:54.446806+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:00:14.432621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640946+0000","last_change":"2024-09-10T04:00:29.138251+0000","last_active":"2024-09-10T04:00:54.640946+0000","last_peered":"2024-09-10T04:00:54.640946+0000","last_clean":"2024-09-10T04:00:54.640946+0000","last_became_active":"2024-09-10T04:00:29.137899+0000","last_became_peered":"2024-09-10T04:00:29.137899+0000","last_unstale":"2024-09-10T04:00:54.640946+0000","last_undegraded":"2024-09-10T04:00:54.640946+0000","last_fullsized":"2024-09-10T04:00:54.640946+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T16:00:02.749682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"70'25","reported_seq":69,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639007+0000","last_change":"2024-09-10T04:00:25.186747+0000","last_active":"2024-09-10T04:00:54.639007+0000","last_peered":"2024-09-10T04:00:54.639007+0000","last_clean":"2024-09-10T04:00:54.639007+0000","last_became_active":"2024-09-10T04:00:25.186345+0000","last_became_peered":"2024-09-10T04:00:25.186345+0000","last_unstale":"2024-09-10T04:00:54.639007+0000","last_undegraded":"2024-09-10T04:00:54.639007+0000","last_fullsized":"2024-09-10T04:00:54.639007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:15:47.379135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":41,"num_read_kb":29,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439846+0000","last_change":"2024-09-10T04:00:23.066328+0000","last_active":"2024-09-10T04:00:54.439846+0000","last_peered":"2024-09-10T04:00:54.439846+0000","last_clean":"2024-09-10T04:00:54.439846+0000","last_became_active":"2024-09-10T04:00:23.065500+0000","last_became_peered":"2024-09-10T04:00:23.065500+0000","last_unstale":"2024-09-10T04:00:54.439846+0000","last_undegraded":"2024-09-10T04:00:54.439846+0000","last_fullsized":"2024-09-10T04:00:54.439846+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:41:42.282547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446863+0000","last_change":"2024-09-10T04:00:27.101424+0000","last_active":"2024-09-10T04:00:54.446863+0000","last_peered":"2024-09-10T04:00:54.446863+0000","last_clean":"2024-09-10T04:00:54.446863+0000","last_became_active":"2024-09-10T04:00:27.101151+0000","last_became_peered":"2024-09-10T04:00:27.101151+0000","last_unstale":"2024-09-10T04:00:54.446863+0000","last_undegraded":"2024-09-10T04:00:54.446863+0000","last_fullsized":"2024-09-10T04:00:54.446863+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:55:36.488752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438112+0000","last_change":"2024-09-10T04:00:29.136574+0000","last_active":"2024-09-10T04:00:54.438112+0000","last_peered":"2024-09-10T04:00:54.438112+0000","last_clean":"2024-09-10T04:00:54.438112+0000","last_became_active":"2024-09-10T04:00:29.136265+0000","last_became_peered":"2024-09-10T04:00:29.136265+0000","last_unstale":"2024-09-10T04:00:54.438112+0000","last_undegraded":"2024-09-10T04:00:54.438112+0000","last_fullsized":"2024-09-10T04:00:54.438112+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:00:41.300222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"69'24","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640618+0000","last_change":"2024-09-10T04:00:25.079652+0000","last_active":"2024-09-10T04:00:54.640618+0000","last_peered":"2024-09-10T04:00:54.640618+0000","last_clean":"2024-09-10T04:00:54.640618+0000","last_became_active":"2024-09-10T04:00:25.079297+0000","last_became_peered":"2024-09-10T04:00:25.079297+0000","last_unstale":"2024-09-10T04:00:54.640618+0000","last_undegraded":"2024-09-10T04:00:54.640618+0000","last_fullsized":"2024-09-10T04:00:54.640618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:44.368142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":32,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438649+0000","last_change":"2024-09-10T04:00:23.075836+0000","last_active":"2024-09-10T04:00:54.438649+0000","last_peered":"2024-09-10T04:00:54.438649+0000","last_clean":"2024-09-10T04:00:54.438649+0000","last_became_active":"2024-09-10T04:00:23.075521+0000","last_became_peered":"2024-09-10T04:00:23.075521+0000","last_unstale":"2024-09-10T04:00:54.438649+0000","last_undegraded":"2024-09-10T04:00:54.438649+0000","last_fullsized":"2024-09-10T04:00:54.438649+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:53:27.895284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.446634+0000","last_change":"2024-09-10T04:00:27.098919+0000","last_active":"2024-09-10T04:00:54.446634+0000","last_peered":"2024-09-10T04:00:54.446634+0000","last_clean":"2024-09-10T04:00:54.446634+0000","last_became_active":"2024-09-10T04:00:27.098606+0000","last_became_peered":"2024-09-10T04:00:27.098606+0000","last_unstale":"2024-09-10T04:00:54.446634+0000","last_undegraded":"2024-09-10T04:00:54.446634+0000","last_fullsized":"2024-09-10T04:00:54.446634+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:37:55.170959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441746+0000","last_change":"2024-09-10T04:00:29.129268+0000","last_active":"2024-09-10T04:00:54.441746+0000","last_peered":"2024-09-10T04:00:54.441746+0000","last_clean":"2024-09-10T04:00:54.441746+0000","last_became_active":"2024-09-10T04:00:29.128919+0000","last_became_peered":"2024-09-10T04:00:29.128919+0000","last_unstale":"2024-09-10T04:00:54.441746+0000","last_undegraded":"2024-09-10T04:00:54.441746+0000","last_fullsized":"2024-09-10T04:00:54.441746+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:39:08.112134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"70'27","reported_seq":75,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638515+0000","last_change":"2024-09-10T04:00:25.186721+0000","last_active":"2024-09-10T04:00:54.638515+0000","last_peered":"2024-09-10T04:00:54.638515+0000","last_clean":"2024-09-10T04:00:54.638515+0000","last_became_active":"2024-09-10T04:00:25.186333+0000","last_became_peered":"2024-09-10T04:00:25.186333+0000","last_unstale":"2024-09-10T04:00:54.638515+0000","last_undegraded":"2024-09-10T04:00:54.638515+0000","last_fullsized":"2024-09-10T04:00:54.638515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:42:28.526051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":44,"num_read_kb":33,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"62'2","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.440163+0000","last_change":"2024-09-10T04:00:23.067471+0000","last_active":"2024-09-10T04:00:54.440163+0000","last_peered":"2024-09-10T04:00:54.440163+0000","last_clean":"2024-09-10T04:00:54.440163+0000","last_became_active":"2024-09-10T04:00:23.066902+0000","last_became_peered":"2024-09-10T04:00:23.066902+0000","last_unstale":"2024-09-10T04:00:54.440163+0000","last_undegraded":"2024-09-10T04:00:54.440163+0000","last_fullsized":"2024-09-10T04:00:54.440163+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:19:33.001710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"69'8","reported_seq":44,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137635+0000","last_change":"2024-09-10T04:00:27.103814+0000","last_active":"2024-09-10T04:01:08.137635+0000","last_peered":"2024-09-10T04:01:08.137635+0000","last_clean":"2024-09-10T04:01:08.137635+0000","last_became_active":"2024-09-10T04:00:27.103485+0000","last_became_peered":"2024-09-10T04:00:27.103485+0000","last_unstale":"2024-09-10T04:01:08.137635+0000","last_undegraded":"2024-09-10T04:01:08.137635+0000","last_fullsized":"2024-09-10T04:01:08.137635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:06:02.786063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638868+0000","last_change":"2024-09-10T04:00:29.243675+0000","last_active":"2024-09-10T04:00:54.638868+0000","last_peered":"2024-09-10T04:00:54.638868+0000","last_clean":"2024-09-10T04:00:54.638868+0000","last_became_active":"2024-09-10T04:00:29.243338+0000","last_became_peered":"2024-09-10T04:00:29.243338+0000","last_unstale":"2024-09-10T04:00:54.638868+0000","last_undegraded":"2024-09-10T04:00:54.638868+0000","last_fullsized":"2024-09-10T04:00:54.638868+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:04:09.257379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"62'1","reported_seq":36,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638790+0000","last_change":"2024-09-10T04:00:23.065261+0000","last_active":"2024-09-10T04:00:54.638790+0000","last_peered":"2024-09-10T04:00:54.638790+0000","last_clean":"2024-09-10T04:00:54.638790+0000","last_became_active":"2024-09-10T04:00:23.064913+0000","last_became_peered":"2024-09-10T04:00:23.064913+0000","last_unstale":"2024-09-10T04:00:54.638790+0000","last_undegraded":"2024-09-10T04:00:54.638790+0000","last_fullsized":"2024-09-10T04:00:54.638790+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:14:19.623116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"70'12","reported_seq":47,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442445+0000","last_change":"2024-09-10T04:00:25.185120+0000","last_active":"2024-09-10T04:00:54.442445+0000","last_peered":"2024-09-10T04:00:54.442445+0000","last_clean":"2024-09-10T04:00:54.442445+0000","last_became_active":"2024-09-10T04:00:25.184828+0000","last_became_peered":"2024-09-10T04:00:25.184828+0000","last_unstale":"2024-09-10T04:00:54.442445+0000","last_undegraded":"2024-09-10T04:00:54.442445+0000","last_fullsized":"2024-09-10T04:00:54.442445+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:30:12.582746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640112+0000","last_change":"2024-09-10T04:00:27.109527+0000","last_active":"2024-09-10T04:00:54.640112+0000","last_peered":"2024-09-10T04:00:54.640112+0000","last_clean":"2024-09-10T04:00:54.640112+0000","last_became_active":"2024-09-10T04:00:27.109206+0000","last_became_peered":"2024-09-10T04:00:27.109206+0000","last_unstale":"2024-09-10T04:00:54.640112+0000","last_undegraded":"2024-09-10T04:00:54.640112+0000","last_fullsized":"2024-09-10T04:00:54.640112+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:01:38.351914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439404+0000","last_change":"2024-09-10T04:00:29.245968+0000","last_active":"2024-09-10T04:00:54.439404+0000","last_peered":"2024-09-10T04:00:54.439404+0000","last_clean":"2024-09-10T04:00:54.439404+0000","last_became_active":"2024-09-10T04:00:29.245244+0000","last_became_peered":"2024-09-10T04:00:29.245244+0000","last_unstale":"2024-09-10T04:00:54.439404+0000","last_undegraded":"2024-09-10T04:00:54.439404+0000","last_fullsized":"2024-09-10T04:00:54.439404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:17:52.641021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"69'21","reported_seq":66,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642964+0000","last_change":"2024-09-10T04:00:25.187123+0000","last_active":"2024-09-10T04:00:54.642964+0000","last_peered":"2024-09-10T04:00:54.642964+0000","last_clean":"2024-09-10T04:00:54.642964+0000","last_became_active":"2024-09-10T04:00:25.186823+0000","last_became_peered":"2024-09-10T04:00:25.186823+0000","last_unstale":"2024-09-10T04:00:54.642964+0000","last_undegraded":"2024-09-10T04:00:54.642964+0000","last_fullsized":"2024-09-10T04:00:54.642964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:35:15.902297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438303+0000","last_change":"2024-09-10T04:00:23.062547+0000","last_active":"2024-09-10T04:00:54.438303+0000","last_peered":"2024-09-10T04:00:54.438303+0000","last_clean":"2024-09-10T04:00:54.438303+0000","last_became_active":"2024-09-10T04:00:23.062127+0000","last_became_peered":"2024-09-10T04:00:23.062127+0000","last_unstale":"2024-09-10T04:00:54.438303+0000","last_undegraded":"2024-09-10T04:00:54.438303+0000","last_fullsized":"2024-09-10T04:00:54.438303+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:38:29.323508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442357+0000","last_change":"2024-09-10T04:00:27.108710+0000","last_active":"2024-09-10T04:00:54.442357+0000","last_peered":"2024-09-10T04:00:54.442357+0000","last_clean":"2024-09-10T04:00:54.442357+0000","last_became_active":"2024-09-10T04:00:27.108447+0000","last_became_peered":"2024-09-10T04:00:27.108447+0000","last_unstale":"2024-09-10T04:00:54.442357+0000","last_undegraded":"2024-09-10T04:00:54.442357+0000","last_fullsized":"2024-09-10T04:00:54.442357+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:54:27.800551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.447001+0000","last_change":"2024-09-10T04:00:29.126663+0000","last_active":"2024-09-10T04:00:54.447001+0000","last_peered":"2024-09-10T04:00:54.447001+0000","last_clean":"2024-09-10T04:00:54.447001+0000","last_became_active":"2024-09-10T04:00:29.126329+0000","last_became_peered":"2024-09-10T04:00:29.126329+0000","last_unstale":"2024-09-10T04:00:54.447001+0000","last_undegraded":"2024-09-10T04:00:54.447001+0000","last_fullsized":"2024-09-10T04:00:54.447001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:33:09.968489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"70'15","reported_seq":54,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640046+0000","last_change":"2024-09-10T04:00:25.187143+0000","last_active":"2024-09-10T04:00:54.640046+0000","last_peered":"2024-09-10T04:00:54.640046+0000","last_clean":"2024-09-10T04:00:54.640046+0000","last_became_active":"2024-09-10T04:00:25.186746+0000","last_became_peered":"2024-09-10T04:00:25.186746+0000","last_unstale":"2024-09-10T04:00:54.640046+0000","last_undegraded":"2024-09-10T04:00:54.640046+0000","last_fullsized":"2024-09-10T04:00:54.640046+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:08:55.309200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":26,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641255+0000","last_change":"2024-09-10T04:00:23.068720+0000","last_active":"2024-09-10T04:00:54.641255+0000","last_peered":"2024-09-10T04:00:54.641255+0000","last_clean":"2024-09-10T04:00:54.641255+0000","last_became_active":"2024-09-10T04:00:23.068420+0000","last_became_peered":"2024-09-10T04:00:23.068420+0000","last_unstale":"2024-09-10T04:00:54.641255+0000","last_undegraded":"2024-09-10T04:00:54.641255+0000","last_fullsized":"2024-09-10T04:00:54.641255+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:02:40.163321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"69'8","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:13.137640+0000","last_change":"2024-09-10T04:00:27.108757+0000","last_active":"2024-09-10T04:01:13.137640+0000","last_peered":"2024-09-10T04:01:13.137640+0000","last_clean":"2024-09-10T04:01:13.137640+0000","last_became_active":"2024-09-10T04:00:27.108425+0000","last_became_peered":"2024-09-10T04:00:27.108425+0000","last_unstale":"2024-09-10T04:01:13.137640+0000","last_undegraded":"2024-09-10T04:01:13.137640+0000","last_fullsized":"2024-09-10T04:01:13.137640+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:17:03.631143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641249+0000","last_change":"2024-09-10T04:00:29.125783+0000","last_active":"2024-09-10T04:00:54.641249+0000","last_peered":"2024-09-10T04:00:54.641249+0000","last_clean":"2024-09-10T04:00:54.641249+0000","last_became_active":"2024-09-10T04:00:29.125484+0000","last_became_peered":"2024-09-10T04:00:29.125484+0000","last_unstale":"2024-09-10T04:00:54.641249+0000","last_undegraded":"2024-09-10T04:00:54.641249+0000","last_fullsized":"2024-09-10T04:00:54.641249+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:17:47.054742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"70'22","reported_seq":65,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.442159+0000","last_change":"2024-09-10T04:00:25.185118+0000","last_active":"2024-09-10T04:00:54.442159+0000","last_peered":"2024-09-10T04:00:54.442159+0000","last_clean":"2024-09-10T04:00:54.442159+0000","last_became_active":"2024-09-10T04:00:25.184775+0000","last_became_peered":"2024-09-10T04:00:25.184775+0000","last_unstale":"2024-09-10T04:00:54.442159+0000","last_undegraded":"2024-09-10T04:00:54.442159+0000","last_fullsized":"2024-09-10T04:00:54.442159+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:24:11.672340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438806+0000","last_change":"2024-09-10T04:00:23.071007+0000","last_active":"2024-09-10T04:00:54.438806+0000","last_peered":"2024-09-10T04:00:54.438806+0000","last_clean":"2024-09-10T04:00:54.438806+0000","last_became_active":"2024-09-10T04:00:23.070698+0000","last_became_peered":"2024-09-10T04:00:23.070698+0000","last_unstale":"2024-09-10T04:00:54.438806+0000","last_undegraded":"2024-09-10T04:00:54.438806+0000","last_fullsized":"2024-09-10T04:00:54.438806+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:51:13.553023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"69'8","reported_seq":41,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:01:08.137636+0000","last_change":"2024-09-10T04:00:27.105320+0000","last_active":"2024-09-10T04:01:08.137636+0000","last_peered":"2024-09-10T04:01:08.137636+0000","last_clean":"2024-09-10T04:01:08.137636+0000","last_became_active":"2024-09-10T04:00:27.105011+0000","last_became_peered":"2024-09-10T04:00:27.105011+0000","last_unstale":"2024-09-10T04:01:08.137636+0000","last_undegraded":"2024-09-10T04:01:08.137636+0000","last_fullsized":"2024-09-10T04:01:08.137636+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:43:54.126227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641791+0000","last_change":"2024-09-10T04:00:29.137672+0000","last_active":"2024-09-10T04:00:54.641791+0000","last_peered":"2024-09-10T04:00:54.641791+0000","last_clean":"2024-09-10T04:00:54.641791+0000","last_became_active":"2024-09-10T04:00:29.137319+0000","last_became_peered":"2024-09-10T04:00:29.137319+0000","last_unstale":"2024-09-10T04:00:54.641791+0000","last_undegraded":"2024-09-10T04:00:54.641791+0000","last_fullsized":"2024-09-10T04:00:54.641791+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:26:44.859748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"70'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640796+0000","last_change":"2024-09-10T04:00:25.079477+0000","last_active":"2024-09-10T04:00:54.640796+0000","last_peered":"2024-09-10T04:00:54.640796+0000","last_clean":"2024-09-10T04:00:54.640796+0000","last_became_active":"2024-09-10T04:00:25.079117+0000","last_became_peered":"2024-09-10T04:00:25.079117+0000","last_unstale":"2024-09-10T04:00:54.640796+0000","last_undegraded":"2024-09-10T04:00:54.640796+0000","last_fullsized":"2024-09-10T04:00:54.640796+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T09:22:51.267018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641100+0000","last_change":"2024-09-10T04:00:23.063490+0000","last_active":"2024-09-10T04:00:54.641100+0000","last_peered":"2024-09-10T04:00:54.641100+0000","last_clean":"2024-09-10T04:00:54.641100+0000","last_became_active":"2024-09-10T04:00:23.063092+0000","last_became_peered":"2024-09-10T04:00:23.063092+0000","last_unstale":"2024-09-10T04:00:54.641100+0000","last_undegraded":"2024-09-10T04:00:54.641100+0000","last_fullsized":"2024-09-10T04:00:54.641100+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T13:15:10.834112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641817+0000","last_change":"2024-09-10T04:00:27.106419+0000","last_active":"2024-09-10T04:00:54.641817+0000","last_peered":"2024-09-10T04:00:54.641817+0000","last_clean":"2024-09-10T04:00:54.641817+0000","last_became_active":"2024-09-10T04:00:27.106125+0000","last_became_peered":"2024-09-10T04:00:27.106125+0000","last_unstale":"2024-09-10T04:00:54.641817+0000","last_undegraded":"2024-09-10T04:00:54.641817+0000","last_fullsized":"2024-09-10T04:00:54.641817+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:25:42.827308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441358+0000","last_change":"2024-09-10T04:00:29.136190+0000","last_active":"2024-09-10T04:00:54.441358+0000","last_peered":"2024-09-10T04:00:54.441358+0000","last_clean":"2024-09-10T04:00:54.441358+0000","last_became_active":"2024-09-10T04:00:29.135889+0000","last_became_peered":"2024-09-10T04:00:29.135889+0000","last_unstale":"2024-09-10T04:00:54.441358+0000","last_undegraded":"2024-09-10T04:00:54.441358+0000","last_fullsized":"2024-09-10T04:00:54.441358+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:20:59.654169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"70'9","reported_seq":42,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642198+0000","last_change":"2024-09-10T04:00:25.075537+0000","last_active":"2024-09-10T04:00:54.642198+0000","last_peered":"2024-09-10T04:00:54.642198+0000","last_clean":"2024-09-10T04:00:54.642198+0000","last_became_active":"2024-09-10T04:00:25.075230+0000","last_became_peered":"2024-09-10T04:00:25.075230+0000","last_unstale":"2024-09-10T04:00:54.642198+0000","last_undegraded":"2024-09-10T04:00:54.642198+0000","last_fullsized":"2024-09-10T04:00:54.642198+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:34:13.774853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.439703+0000","last_change":"2024-09-10T04:00:23.067084+0000","last_active":"2024-09-10T04:00:54.439703+0000","last_peered":"2024-09-10T04:00:54.439703+0000","last_clean":"2024-09-10T04:00:54.439703+0000","last_became_active":"2024-09-10T04:00:23.066772+0000","last_became_peered":"2024-09-10T04:00:23.066772+0000","last_unstale":"2024-09-10T04:00:54.439703+0000","last_undegraded":"2024-09-10T04:00:54.439703+0000","last_fullsized":"2024-09-10T04:00:54.439703+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:55:59.182575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441839+0000","last_change":"2024-09-10T04:00:27.104865+0000","last_active":"2024-09-10T04:00:54.441839+0000","last_peered":"2024-09-10T04:00:54.441839+0000","last_clean":"2024-09-10T04:00:54.441839+0000","last_became_active":"2024-09-10T04:00:27.104594+0000","last_became_peered":"2024-09-10T04:00:27.104594+0000","last_unstale":"2024-09-10T04:00:54.441839+0000","last_undegraded":"2024-09-10T04:00:54.441839+0000","last_fullsized":"2024-09-10T04:00:54.441839+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T06:48:51.304827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639413+0000","last_change":"2024-09-10T04:00:29.139566+0000","last_active":"2024-09-10T04:00:54.639413+0000","last_peered":"2024-09-10T04:00:54.639413+0000","last_clean":"2024-09-10T04:00:54.639413+0000","last_became_active":"2024-09-10T04:00:29.139228+0000","last_became_peered":"2024-09-10T04:00:29.139228+0000","last_unstale":"2024-09-10T04:00:54.639413+0000","last_undegraded":"2024-09-10T04:00:54.639413+0000","last_fullsized":"2024-09-10T04:00:54.639413+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T11:56:28.697922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"70'18","reported_seq":58,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441495+0000","last_change":"2024-09-10T04:00:25.185154+0000","last_active":"2024-09-10T04:00:54.441495+0000","last_peered":"2024-09-10T04:00:54.441495+0000","last_clean":"2024-09-10T04:00:54.441495+0000","last_became_active":"2024-09-10T04:00:25.184863+0000","last_became_peered":"2024-09-10T04:00:25.184863+0000","last_unstale":"2024-09-10T04:00:54.441495+0000","last_undegraded":"2024-09-10T04:00:54.441495+0000","last_fullsized":"2024-09-10T04:00:54.441495+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:05:16.314765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.640203+0000","last_change":"2024-09-10T04:00:23.075927+0000","last_active":"2024-09-10T04:00:54.640203+0000","last_peered":"2024-09-10T04:00:54.640203+0000","last_clean":"2024-09-10T04:00:54.640203+0000","last_became_active":"2024-09-10T04:00:23.075534+0000","last_became_peered":"2024-09-10T04:00:23.075534+0000","last_unstale":"2024-09-10T04:00:54.640203+0000","last_undegraded":"2024-09-10T04:00:54.640203+0000","last_fullsized":"2024-09-10T04:00:54.640203+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T14:10:48.618323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.437867+0000","last_change":"2024-09-10T04:00:27.104739+0000","last_active":"2024-09-10T04:00:54.437867+0000","last_peered":"2024-09-10T04:00:54.437867+0000","last_clean":"2024-09-10T04:00:54.437867+0000","last_became_active":"2024-09-10T04:00:27.104205+0000","last_became_peered":"2024-09-10T04:00:27.104205+0000","last_unstale":"2024-09-10T04:00:54.437867+0000","last_undegraded":"2024-09-10T04:00:54.437867+0000","last_fullsized":"2024-09-10T04:00:54.437867+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T15:02:04.153702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441922+0000","last_change":"2024-09-10T04:00:29.243928+0000","last_active":"2024-09-10T04:00:54.441922+0000","last_peered":"2024-09-10T04:00:54.441922+0000","last_clean":"2024-09-10T04:00:54.441922+0000","last_became_active":"2024-09-10T04:00:29.243545+0000","last_became_peered":"2024-09-10T04:00:29.243545+0000","last_unstale":"2024-09-10T04:00:54.441922+0000","last_undegraded":"2024-09-10T04:00:54.441922+0000","last_fullsized":"2024-09-10T04:00:54.441922+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T05:09:14.153640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"69'4","reported_seq":32,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441368+0000","last_change":"2024-09-10T04:00:25.079602+0000","last_active":"2024-09-10T04:00:54.441368+0000","last_peered":"2024-09-10T04:00:54.441368+0000","last_clean":"2024-09-10T04:00:54.441368+0000","last_became_active":"2024-09-10T04:00:25.079300+0000","last_became_peered":"2024-09-10T04:00:25.079300+0000","last_unstale":"2024-09-10T04:00:54.441368+0000","last_undegraded":"2024-09-10T04:00:54.441368+0000","last_fullsized":"2024-09-10T04:00:54.441368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:02:45.390334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641041+0000","last_change":"2024-09-10T04:00:23.069048+0000","last_active":"2024-09-10T04:00:54.641041+0000","last_peered":"2024-09-10T04:00:54.641041+0000","last_clean":"2024-09-10T04:00:54.641041+0000","last_became_active":"2024-09-10T04:00:23.068792+0000","last_became_peered":"2024-09-10T04:00:23.068792+0000","last_unstale":"2024-09-10T04:00:54.641041+0000","last_undegraded":"2024-09-10T04:00:54.641041+0000","last_fullsized":"2024-09-10T04:00:54.641041+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T12:04:57.535680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.638674+0000","last_change":"2024-09-10T04:00:27.107361+0000","last_active":"2024-09-10T04:00:54.638674+0000","last_peered":"2024-09-10T04:00:54.638674+0000","last_clean":"2024-09-10T04:00:54.638674+0000","last_became_active":"2024-09-10T04:00:27.107078+0000","last_became_peered":"2024-09-10T04:00:27.107078+0000","last_unstale":"2024-09-10T04:00:54.638674+0000","last_undegraded":"2024-09-10T04:00:54.638674+0000","last_fullsized":"2024-09-10T04:00:54.638674+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:03:44.442007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.438121+0000","last_change":"2024-09-10T04:00:29.128837+0000","last_active":"2024-09-10T04:00:54.438121+0000","last_peered":"2024-09-10T04:00:54.438121+0000","last_clean":"2024-09-10T04:00:54.438121+0000","last_became_active":"2024-09-10T04:00:29.128517+0000","last_became_peered":"2024-09-10T04:00:29.128517+0000","last_unstale":"2024-09-10T04:00:54.438121+0000","last_undegraded":"2024-09-10T04:00:54.438121+0000","last_fullsized":"2024-09-10T04:00:54.438121+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T10:23:42.488602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":18,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.642606+0000","last_change":"2024-09-10T04:00:29.132840+0000","last_active":"2024-09-10T04:00:54.642606+0000","last_peered":"2024-09-10T04:00:54.642606+0000","last_clean":"2024-09-10T04:00:54.642606+0000","last_became_active":"2024-09-10T04:00:29.132467+0000","last_became_peered":"2024-09-10T04:00:29.132467+0000","last_unstale":"2024-09-10T04:00:54.642606+0000","last_undegraded":"2024-09-10T04:00:54.642606+0000","last_fullsized":"2024-09-10T04:00:54.642606+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:28.096235+0000","last_clean_scrub_stamp":"2024-09-10T04:00:28.096235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:00:01.943463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":30,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.441645+0000","last_change":"2024-09-10T04:00:23.077521+0000","last_active":"2024-09-10T04:00:54.441645+0000","last_peered":"2024-09-10T04:00:54.441645+0000","last_clean":"2024-09-10T04:00:54.441645+0000","last_became_active":"2024-09-10T04:00:23.077243+0000","last_became_peered":"2024-09-10T04:00:23.077243+0000","last_unstale":"2024-09-10T04:00:54.441645+0000","last_undegraded":"2024-09-10T04:00:54.441645+0000","last_fullsized":"2024-09-10T04:00:54.441645+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:22.018197+0000","last_clean_scrub_stamp":"2024-09-10T04:00:22.018197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T04:14:01.098627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"70'17","reported_seq":57,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.639800+0000","last_change":"2024-09-10T04:00:25.074605+0000","last_active":"2024-09-10T04:00:54.639800+0000","last_peered":"2024-09-10T04:00:54.639800+0000","last_clean":"2024-09-10T04:00:54.639800+0000","last_became_active":"2024-09-10T04:00:25.074281+0000","last_became_peered":"2024-09-10T04:00:25.074281+0000","last_unstale":"2024-09-10T04:00:54.639800+0000","last_undegraded":"2024-09-10T04:00:54.639800+0000","last_fullsized":"2024-09-10T04:00:54.639800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:24.051189+0000","last_clean_scrub_stamp":"2024-09-10T04:00:24.051189+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T07:54:19.219164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":29,"num_read_kb":21,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":22,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T04:00:54.641930+0000","last_change":"2024-09-10T04:00:27.100686+0000","last_active":"2024-09-10T04:00:54.641930+0000","last_peered":"2024-09-10T04:00:54.641930+0000","last_clean":"2024-09-10T04:00:54.641930+0000","last_became_active":"2024-09-10T04:00:27.100382+0000","last_became_peered":"2024-09-10T04:00:27.100382+0000","last_unstale":"2024-09-10T04:00:54.641930+0000","last_undegraded":"2024-09-10T04:00:54.641930+0000","last_fullsized":"2024-09-10T04:00:54.641930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T04:00:26.057924+0000","last_clean_scrub_stamp":"2024-09-10T04:00:26.057924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T08:34:46.630185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":210,"num_object_clones":0,"num_object_copies":630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":210,"num_whiteouts":0,"num_read":986,"num_read_kb":711,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":586,"ondisk_log_size":586,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":47,"num_read_kb":42,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201296,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28376,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712680,"statfs":{"total":95990841344,"available":95961784320,"internally_reserved":0,"allocated":1384448,"data_stored":916542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462934,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":28364,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93712692,"statfs":{"total":95990841344,"available":95961796608,"internally_reserved":0,"allocated":1372160,"data_stored":916886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724571,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31948,"kb_used_data":760,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93709108,"statfs":{"total":95990841344,"available":95958126592,"internally_reserved":0,"allocated":778240,"data_stored":325884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986208,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27812,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713244,"statfs":{"total":95990841344,"available":95962361856,"internally_reserved":0,"allocated":802816,"data_stored":325707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":31976,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93709080,"statfs":{"total":95990841344,"available":95958097920,"internally_reserved":0,"allocated":806912,"data_stored":325980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27784,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713272,"statfs":{"total":95990841344,"available":95962390528,"internally_reserved":0,"allocated":774144,"data_stored":326243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93713204,"statfs":{"total":95990841344,"available":95962320896,"internally_reserved":0,"allocated":843776,"data_stored":327880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32544,"kb_used_data":1360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708512,"statfs":{"total":95990841344,"available":95957516288,"internally_reserved":0,"allocated":1392640,"data_stored":916687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T04:01:21.685 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T04:01:21.685 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T04:01:21.685 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T04:01:21.685 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T04:01:22.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:21 smithi050 ceph-mon[26687]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:22.254 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:21 smithi038 ceph-mon[30767]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:22.254 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:21 smithi038 ceph-mon[25393]: from='client.24629 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T04:01:23.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:22 smithi038 ceph-mon[25393]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:23.212 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:22 smithi038 ceph-mon[30767]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:23.222 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:23.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:22 smithi050 ceph-mon[26687]: pgmap v17: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:24.411 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-10T04:01:24.411 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T04:01:24.462 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:24 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ::ffff:172.21.15.50 - - [10/Sep/2024:04:01:24] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-10T04:01:24.554 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:24 smithi050 ceph-mon[26687]: from='client.? 172.21.15.38:0/4018514129' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T04:01:24.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:24 smithi038 ceph-mon[25393]: from='client.? 172.21.15.38:0/4018514129' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T04:01:24.754 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:24 smithi038 ceph-mon[30767]: from='client.? 172.21.15.38:0/4018514129' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-10T04:01:25.253 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T04:01:25.253 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T04:01:25.254 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T04:01:25.264 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-10T04:01:25.264 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-10T04:01:25.776 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:25.804 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:01:25 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:01:25.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:25 smithi050 ceph-mon[26687]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:25.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:25 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:25.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:25 smithi038 ceph-mon[25393]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:25.824 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:25 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:25.824 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:25 smithi038 ceph-mon[30767]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:25.824 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:25 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:26.261 INFO:teuthology.orchestra.run.smithi038.stdout:167 167 2024-09-10T04:01:26.766 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:26 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:26.957 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:26 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:26.957 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:26 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:26.989 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T04:01:27.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:27 smithi038 ceph-mon[25393]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:27.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:27 smithi038 ceph-mon[30767]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:27.755 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:01:27 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:01:27.464Z caller=cluster.go:700 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001907432s 2024-09-10T04:01:27.772 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:27 smithi050 ceph-mon[26687]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:27.804 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:28.403 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-09-10T04:01:28.403 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-09-10T04:01:28.403 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-09-10T04:01:29.136 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T04:01:29.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 systemd[1]: Starting Ceph grafana.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:29.398 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:29.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-mon[26687]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:29.633 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-mon[26687]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:29.633 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 podman[48570]: 2024-09-10 04:01:29.27905455 +0000 UTC m=+0.017499206 image pull c8b91775d855b99270fc5d22f3c6737e8cca01ef4c25c8b0362295e0746fa39b quay.io/ceph/grafana:10.4.0 2024-09-10T04:01:29.633 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 podman[48570]: 2024-09-10 04:01:29.405250742 +0000 UTC m=+0.143695436 container create 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:01:29.633 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 podman[48570]: 2024-09-10 04:01:29.632517919 +0000 UTC m=+0.370962566 container init 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:01:29.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:29 smithi038 ceph-mon[25393]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:29.755 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:29 smithi038 ceph-mon[25393]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:29 smithi038 ceph-mon[30767]: from='client.24641 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:29.755 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:29 smithi038 ceph-mon[30767]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:29.885 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 podman[48570]: 2024-09-10 04:01:29.635632653 +0000 UTC m=+0.374077300 container start 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:01:29.885 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 bash[48570]: 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 systemd[1]: Started Ceph grafana.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765571285Z level=info msg="Starting Grafana" version=10.4.0 commit=03f502a94d17f7dc4e6c34acdf8428aedd986e4c branch=HEAD compiled=2024-09-10T04:01:29Z 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765852862Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765861326Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765865238Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765868383Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-10T04:01:29.886 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765871239Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.76587469Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765878026Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765881449Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765884502Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765887298Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765890142Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.76589336Z level=info msg=Target target=[all] 2024-09-10T04:01:29.887 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765900435Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765903246Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765905876Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765908502Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765911337Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=settings t=2024-09-10T04:01:29.765914095Z level=info msg="App mode production" 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore t=2024-09-10T04:01:29.766165604Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T04:01:29.888 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore t=2024-09-10T04:01:29.766179792Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.766644978Z level=info msg="Starting DB migrations" 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.767537122Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.768325878Z level=info msg="Migration successfully executed" id="create migration_log table" duration=788.343µs 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.768823318Z level=info msg="Executing migration" id="create user table" 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.769373207Z level=info msg="Migration successfully executed" id="create user table" duration=551.202µs 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.769770143Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.77025002Z level=info msg="Migration successfully executed" id="add unique index user.login" duration=479.708µs 2024-09-10T04:01:29.889 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.77060283Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.77106665Z level=info msg="Migration successfully executed" id="add unique index user.email" duration=464.923µs 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.771407494Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.771868851Z level=info msg="Migration successfully executed" id="drop index UQE_user_login - v1" duration=461.401µs 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.772257087Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.772718169Z level=info msg="Migration successfully executed" id="drop index UQE_user_email - v1" duration=460.711µs 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.773033123Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T04:01:29.890 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.774697536Z level=info msg="Migration successfully executed" id="Rename table user to user_v1 - v1" duration=1.665318ms 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.775045513Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.775576719Z level=info msg="Migration successfully executed" id="create user table v2" duration=531.018µs 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.775884154Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.776333701Z level=info msg="Migration successfully executed" id="create index UQE_user_login - v2" duration=449.229µs 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.776679115Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T04:01:29.891 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.777122682Z level=info msg="Migration successfully executed" id="create index UQE_user_email - v2" duration=442.553µs 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.777497769Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.777760383Z level=info msg="Migration successfully executed" id="copy data_source v1 to v2" duration=262.44µs 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.778055311Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.77842423Z level=info msg="Migration successfully executed" id="Drop old table user_v1" duration=368.763µs 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.778743537Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.779491478Z level=info msg="Migration successfully executed" id="Add column help_flags1 to user table" duration=747.603µs 2024-09-10T04:01:29.892 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.779867031Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T04:01:29.893 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.779888045Z level=info msg="Migration successfully executed" id="Update user table charset" duration=21.816µs 2024-09-10T04:01:29.893 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.780386638Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T04:01:29.893 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.781387459Z level=info msg="Migration successfully executed" id="Add last_seen_at column to user" duration=1.000118ms 2024-09-10T04:01:29.893 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.781794593Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T04:01:29.896 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.781975876Z level=info msg="Migration successfully executed" id="Add missing user data" duration=181.034µs 2024-09-10T04:01:29.896 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.782486865Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T04:01:29.896 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.783516261Z level=info msg="Migration successfully executed" id="Add is_disabled column to user" duration=1.0288ms 2024-09-10T04:01:29.896 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.783979546Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.784482418Z level=info msg="Migration successfully executed" id="Add index user.login/user.email" duration=503.426µs 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.784804644Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.785563138Z level=info msg="Migration successfully executed" id="Add is_service_account column to user" duration=759.662µs 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.7859225Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.791668309Z level=info msg="Migration successfully executed" id="Update is_service_account column to nullable" duration=5.745438ms 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.792023623Z level=info msg="Executing migration" id="Add uid column to user" 2024-09-10T04:01:29.897 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.792853172Z level=info msg="Migration successfully executed" id="Add uid column to user" duration=829.831µs 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.793178801Z level=info msg="Executing migration" id="Update uid column values for users" 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.793334541Z level=info msg="Migration successfully executed" id="Update uid column values for users" duration=155.054µs 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.793751628Z level=info msg="Executing migration" id="Add unique index user_uid" 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.794210067Z level=info msg="Migration successfully executed" id="Add unique index user_uid" duration=458.428µs 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.794612469Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.795123703Z level=info msg="Migration successfully executed" id="create temp user table v1-7" duration=510.905µs 2024-09-10T04:01:29.898 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.795493619Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.795945016Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v1-7" duration=451.279µs 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.796279779Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.796732672Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v1-7" duration=452.724µs 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.797064109Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.797527317Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v1-7" duration=461.687µs 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.797860508Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.798310267Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v1-7" duration=449.514µs 2024-09-10T04:01:29.899 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.79865614Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.798673917Z level=info msg="Migration successfully executed" id="Update temp_user table charset" duration=18.417µs 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.799052558Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.799511435Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_email - v1" duration=458.7µs 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.799842057Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.80029256Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_org_id - v1" duration=448.965µs 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.800610295Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.801069418Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_code - v1" duration=459.102µs 2024-09-10T04:01:29.900 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.80138274Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.801833448Z level=info msg="Migration successfully executed" id="drop index IDX_temp_user_status - v1" duration=450.641µs 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.802154367Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.804426565Z level=info msg="Migration successfully executed" id="Rename table temp_user to temp_user_tmp_qwerty - v1" duration=2.271498ms 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.804770316Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.805305374Z level=info msg="Migration successfully executed" id="create temp_user v2" duration=533.026µs 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.80562229Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.806089621Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_email - v2" duration=467.156µs 2024-09-10T04:01:29.901 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.806406941Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.806878457Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_org_id - v2" duration=472.418µs 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.807191101Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.807664552Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_code - v2" duration=474.02µs 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.807966022Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.808423069Z level=info msg="Migration successfully executed" id="create index IDX_temp_user_status - v2" duration=456.916µs 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.80876969Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.809027284Z level=info msg="Migration successfully executed" id="copy temp_user v1 to v2" duration=257.355µs 2024-09-10T04:01:29.902 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.809318552Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.80969574Z level=info msg="Migration successfully executed" id="drop temp_user_tmp_qwerty" duration=377.025µs 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.809991672Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.810232294Z level=info msg="Migration successfully executed" id="Set created for temp users that will otherwise prematurely expire" duration=240.661µs 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.810564927Z level=info msg="Executing migration" id="create star table" 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.810975559Z level=info msg="Migration successfully executed" id="create star table" duration=410.338µs 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.811275113Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.811752558Z level=info msg="Migration successfully executed" id="add unique index star.user_id_dashboard_id" duration=476.349µs 2024-09-10T04:01:29.903 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.812086769Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T04:01:29.904 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.812597851Z level=info msg="Migration successfully executed" id="create org table v1" duration=510.978µs 2024-09-10T04:01:29.904 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.812931789Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T04:01:29.904 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.813419112Z level=info msg="Migration successfully executed" id="create index UQE_org_name - v1" duration=486.99µs 2024-09-10T04:01:29.904 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.81383741Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T04:01:29.905 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.814287197Z level=info msg="Migration successfully executed" id="create org_user table v1" duration=449.738µs 2024-09-10T04:01:29.905 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.814655195Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T04:01:29.905 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.815171327Z level=info msg="Migration successfully executed" id="create index IDX_org_user_org_id - v1" duration=516.081µs 2024-09-10T04:01:29.905 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.815488944Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T04:01:29.905 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.815987956Z level=info msg="Migration successfully executed" id="create index UQE_org_user_org_id_user_id - v1" duration=498.81µs 2024-09-10T04:01:29.908 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.816300989Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T04:01:29.908 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.816791549Z level=info msg="Migration successfully executed" id="create index IDX_org_user_user_id - v1" duration=490.398µs 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.817099889Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.817116515Z level=info msg="Migration successfully executed" id="Update org table charset" duration=18.386µs 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.81748156Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.81749724Z level=info msg="Migration successfully executed" id="Update org_user table charset" duration=16.47µs 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.817856901Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T04:01:29.909 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.817963665Z level=info msg="Migration successfully executed" id="Migrate all Read Only Viewers to Viewers" duration=106.762µs 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.81828088Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.818831549Z level=info msg="Migration successfully executed" id="create dashboard table" duration=550.369µs 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.819147841Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.819679313Z level=info msg="Migration successfully executed" id="add index dashboard.account_id" duration=531.232µs 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.820012946Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.820541557Z level=info msg="Migration successfully executed" id="add unique index dashboard_account_id_slug" duration=528.377µs 2024-09-10T04:01:29.910 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.820964184Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.821406304Z level=info msg="Migration successfully executed" id="create dashboard_tag table" duration=442.1µs 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.821759863Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.822256694Z level=info msg="Migration successfully executed" id="add unique index dashboard_tag.dasboard_id_term" duration=496.781µs 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.822592317Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.823070034Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" duration=477.722µs 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.823374005Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.827148406Z level=info msg="Migration successfully executed" id="Rename table dashboard to dashboard_v1 - v1" duration=3.771077ms 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.827619853Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T04:01:29.911 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.828199795Z level=info msg="Migration successfully executed" id="create dashboard v2" duration=580.028µs 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.828562727Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.829050487Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_org_id - v2" duration=487.758µs 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.829381015Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.829882028Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_org_id_slug - v2" duration=500.752µs 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.830234509Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.830490971Z level=info msg="Migration successfully executed" id="copy dashboard v1 to v2" duration=256.276µs 2024-09-10T04:01:29.912 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.830795017Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.831452948Z level=info msg="Migration successfully executed" id="drop table dashboard_v1" duration=656.224µs 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.831782713Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.831823556Z level=info msg="Migration successfully executed" id="alter dashboard.data to mediumtext v1" duration=41.176µs 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.832207673Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.833406647Z level=info msg="Migration successfully executed" id="Add column updated_by in dashboard - v2" duration=1.198522ms 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.833746083Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.834905651Z level=info msg="Migration successfully executed" id="Add column created_by in dashboard - v2" duration=1.159219ms 2024-09-10T04:01:29.913 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.835232277Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T04:01:29.914 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.836397798Z level=info msg="Migration successfully executed" id="Add column gnetId in dashboard" duration=1.165361ms 2024-09-10T04:01:29.914 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.836841746Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T04:01:29.914 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.837433585Z level=info msg="Migration successfully executed" id="Add index for gnetId in dashboard" duration=593.912µs 2024-09-10T04:01:29.914 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.837770525Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T04:01:29.914 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.838989814Z level=info msg="Migration successfully executed" id="Add column plugin_id in dashboard" duration=1.218993ms 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.839310857Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.839824093Z level=info msg="Migration successfully executed" id="Add index for plugin_id in dashboard" duration=513.118µs 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.840146776Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.840785063Z level=info msg="Migration successfully executed" id="Add index for dashboard_id in dashboard_tag" duration=637.057µs 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.841169784Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.841187078Z level=info msg="Migration successfully executed" id="Update dashboard table charset" duration=17.996µs 2024-09-10T04:01:29.915 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.84158929Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.841607922Z level=info msg="Migration successfully executed" id="Update dashboard_tag table charset" duration=19.236µs 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.84200686Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.843562806Z level=info msg="Migration successfully executed" id="Add column folder_id in dashboard" duration=1.553274ms 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.843906807Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.845140684Z level=info msg="Migration successfully executed" id="Add column isFolder in dashboard" duration=1.233581ms 2024-09-10T04:01:29.916 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.845762356Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T04:01:29.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.847171037Z level=info msg="Migration successfully executed" id="Add column has_acl in dashboard" duration=1.408376ms 2024-09-10T04:01:29.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.847512283Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T04:01:29.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.848806341Z level=info msg="Migration successfully executed" id="Add column uid in dashboard" duration=1.293734ms 2024-09-10T04:01:29.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.849117439Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T04:01:29.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.849265132Z level=info msg="Migration successfully executed" id="Update uid column values in dashboard" duration=147.717µs 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.849665542Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.850156849Z level=info msg="Migration successfully executed" id="Add unique index dashboard_org_id_uid" duration=489.967µs 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.850487156Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.850988832Z level=info msg="Migration successfully executed" id="Remove unique index org_id_slug" duration=501.488µs 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.851294312Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T04:01:29.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.851311401Z level=info msg="Migration successfully executed" id="Update dashboard title length" duration=17.938µs 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.851743711Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.852547996Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_title_folder_id" duration=802.321µs 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.853024404Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.853734687Z level=info msg="Migration successfully executed" id="create dashboard_provisioning" duration=709.996µs 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.85418667Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T04:01:29.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.859892797Z level=info msg="Migration successfully executed" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" duration=5.705686ms 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.860332849Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.860969348Z level=info msg="Migration successfully executed" id="create dashboard_provisioning v2" duration=635.074µs 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.862116092Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.862688715Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id - v2" duration=573.086µs 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.863023756Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T04:01:29.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.863562702Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" duration=538.687µs 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.86388535Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.864115039Z level=info msg="Migration successfully executed" id="copy dashboard_provisioning v1 to v2" duration=229.663µs 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.86441819Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.864862463Z level=info msg="Migration successfully executed" id="drop dashboard_provisioning_tmp_qwerty" duration=444.158µs 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.865169412Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T04:01:29.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.866447895Z level=info msg="Migration successfully executed" id="Add check_sum column" duration=1.278216ms 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.866776315Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.867270797Z level=info msg="Migration successfully executed" id="Add index for dashboard_title" duration=492.919µs 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.867577614Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.867696026Z level=info msg="Migration successfully executed" id="delete tags for deleted dashboards" duration=117.729µs 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.868046916Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T04:01:29.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.868155624Z level=info msg="Migration successfully executed" id="delete stars for deleted dashboards" duration=109.091µs 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.868486947Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.868988698Z level=info msg="Migration successfully executed" id="Add index for dashboard_is_folder" duration=501.442µs 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.869302045Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.870651684Z level=info msg="Migration successfully executed" id="Add isPublic for dashboard" duration=1.348252ms 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.870983476Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.871617286Z level=info msg="Migration successfully executed" id="create data_source table" duration=633.355µs 2024-09-10T04:01:29.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.87198474Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T04:01:29.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.872521859Z level=info msg="Migration successfully executed" id="add index data_source.account_id" duration=538.316µs 2024-09-10T04:01:29.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.87284207Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T04:01:29.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.873351207Z level=info msg="Migration successfully executed" id="add unique index data_source.account_id_name" duration=508.888µs 2024-09-10T04:01:29.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.873676355Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T04:01:29.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.874174205Z level=info msg="Migration successfully executed" id="drop index IDX_data_source_account_id - v1" duration=497.627µs 2024-09-10T04:01:29.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.874483065Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T04:01:29.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.874978722Z level=info msg="Migration successfully executed" id="drop index UQE_data_source_account_id_name - v1" duration=495.562µs 2024-09-10T04:01:29.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.875282224Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T04:01:29.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.878920152Z level=info msg="Migration successfully executed" id="Rename table data_source to data_source_v1 - v1" duration=3.637664ms 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.879260674Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.879864156Z level=info msg="Migration successfully executed" id="create data_source table v2" duration=603.239µs 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.880185917Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.880734373Z level=info msg="Migration successfully executed" id="create index IDX_data_source_org_id - v2" duration=548.12µs 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.881049259Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T04:01:29.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.881606727Z level=info msg="Migration successfully executed" id="create index UQE_data_source_org_id_name - v2" duration=557.171µs 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.881932277Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.882323298Z level=info msg="Migration successfully executed" id="Drop old table data_source_v1 #2" duration=389.616µs 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.882652091Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.884161331Z level=info msg="Migration successfully executed" id="Add column with_credentials" duration=1.508862ms 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.884602058Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T04:01:29.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.886415579Z level=info msg="Migration successfully executed" id="Add secure json data column" duration=1.813813ms 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.886746071Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.886762131Z level=info msg="Migration successfully executed" id="Update data_source table charset" duration=16.569µs 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.887145711Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.887275834Z level=info msg="Migration successfully executed" id="Update initial version to 1" duration=130.096µs 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.887664965Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T04:01:29.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.889160191Z level=info msg="Migration successfully executed" id="Add read_only data column" duration=1.494913ms 2024-09-10T04:01:29.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.889486878Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T04:01:29.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.889619975Z level=info msg="Migration successfully executed" id="Migrate logging ds to loki ds" duration=133.162µs 2024-09-10T04:01:29.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.889982312Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T04:01:29.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.890099875Z level=info msg="Migration successfully executed" id="Update json_data with nulls" duration=117.565µs 2024-09-10T04:01:30.066 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:agent.smithi038 smithi038 running 42s ago 5m - - 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:agent.smithi050 smithi050 running 49s ago 5m - - 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager.a smithi038 *:9093,9094 starting - - - - 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:grafana.a smithi050 *:3000 starting - - - - 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:iscsi.iscsi.a smithi050 *:5000 running (56s) 49s ago 55s - - 3.8 d6d839bc8a4b 53bc3ecbe257 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.x smithi050 *:8443,8765 running (5m) 49s ago 5m - - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b f5e92250f36e 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.y smithi038 *:9283,8765 running (6m) 42s ago 6m 522M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 6741900d5e6b 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:mon.a smithi038 running (6m) 42s ago 7m 48.8M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b c626664d8371 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:mon.b smithi050 running (5m) 49s ago 5m - 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 432ee69f7770 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:mon.c smithi038 running (5m) 42s ago 5m 36.7M 2048M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 891472313ee9 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (4m) 42s ago 4m 53.2M 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 641be81577e2 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi038 running (4m) 42s ago 4m 46.5M 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b a70f4bfcec71 2024-09-10T04:01:30.067 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi038 running (3m) 42s ago 3m 43.7M 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 3e4de0aa9497 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi038 running (3m) 42s ago 3m 51.5M 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b bddd0e12876c 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi050 running (2m) 49s ago 2m - 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b ddd2d2153b25 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi050 running (2m) 49s ago 2m - 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 16f82ad450c7 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 smithi050 running (114s) 49s ago 113s - 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b c23f2e78118e 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 smithi050 running (85s) 49s ago 84s - 4096M 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 00103621eda5 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus.a smithi050 *:9095 starting - - - - 2024-09-10T04:01:30.068 INFO:teuthology.orchestra.run.smithi038.stdout:rgw.foo.a smithi038 *:80 running (69s) 42s ago 68s 77.0M - 19.3.0-4885-g3a5e3f16 d6d839bc8a4b 8d456acce75d 2024-09-10T04:01:30.135 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.890436877Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T04:01:30.135 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.891954812Z level=info msg="Migration successfully executed" id="Add uid column" duration=1.5175ms 2024-09-10T04:01:30.135 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.892693291Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.892870706Z level=info msg="Migration successfully executed" id="Update uid value" duration=177.554µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.893363757Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.893968128Z level=info msg="Migration successfully executed" id="Add unique index datasource_org_id_uid" duration=604.454µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.894279711Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.89478384Z level=info msg="Migration successfully executed" id="add unique index datasource_org_id_is_default" duration=503.234µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.895109617Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.895659751Z level=info msg="Migration successfully executed" id="create api_key table" duration=550.002µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.895986815Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.896493838Z level=info msg="Migration successfully executed" id="add index api_key.account_id" duration=506.842µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.896811289Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.897300732Z level=info msg="Migration successfully executed" id="add index api_key.key" duration=489.188µs 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.897668909Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T04:01:30.136 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.898196307Z level=info msg="Migration successfully executed" id="add index api_key.account_id_name" duration=527.099µs 2024-09-10T04:01:30.137 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.898537318Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T04:01:30.137 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.899030461Z level=info msg="Migration successfully executed" id="drop index IDX_api_key_account_id - v1" duration=493.122µs 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.899335822Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.899833319Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_key - v1" duration=497.348µs 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.900165358Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.900663505Z level=info msg="Migration successfully executed" id="drop index UQE_api_key_account_id_name - v1" duration=496.529µs 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.900972194Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.904973142Z level=info msg="Migration successfully executed" id="Rename table api_key to api_key_v1 - v1" duration=4.00063ms 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.905304186Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.905812226Z level=info msg="Migration successfully executed" id="create api_key table v2" duration=507.801µs 2024-09-10T04:01:30.138 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.906127896Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.906633921Z level=info msg="Migration successfully executed" id="create index IDX_api_key_org_id - v2" duration=507.001µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.906953614Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.907437389Z level=info msg="Migration successfully executed" id="create index UQE_api_key_key - v2" duration=483.606µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.907758477Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.908248186Z level=info msg="Migration successfully executed" id="create index UQE_api_key_org_id_name - v2" duration=489.36µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.908571207Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.908815456Z level=info msg="Migration successfully executed" id="copy api_key v1 to v2" duration=244.32µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.909273589Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.909826445Z level=info msg="Migration successfully executed" id="Drop old table api_key_v1" duration=553.23µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.910248592Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.910269126Z level=info msg="Migration successfully executed" id="Update api_key table charset" duration=21.231µs 2024-09-10T04:01:30.139 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.910680465Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.912232746Z level=info msg="Migration successfully executed" id="Add expires to api_key table" duration=1.552263ms 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.912566346Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.914121348Z level=info msg="Migration successfully executed" id="Add service account foreign key" duration=1.554708ms 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.914473594Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.914579227Z level=info msg="Migration successfully executed" id="set service account foreign key to nil if 0" duration=105.791µs 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.914961956Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.916571382Z level=info msg="Migration successfully executed" id="Add last_used_at to api_key table" duration=1.609185ms 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.916912622Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.918550625Z level=info msg="Migration successfully executed" id="Add is_revoked column to api_key table" duration=1.63785ms 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.918867013Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.919369698Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v4" duration=502.554µs 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.919692265Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T04:01:30.140 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.920075971Z level=info msg="Migration successfully executed" id="drop table dashboard_snapshot_v4 #1" duration=384.671µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.920383127Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.920971129Z level=info msg="Migration successfully executed" id="create dashboard_snapshot table v5 #2" duration=586.257µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.921279282Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.921814799Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_key - v5" duration=535.252µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.922133362Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.922687773Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_snapshot_delete_key - v5" duration=553.846µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.923031764Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.9235645Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_snapshot_user_id - v5" duration=532.498µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.923882044Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.92392165Z level=info msg="Migration successfully executed" id="alter dashboard_snapshot to mediumtext v2" duration=39.832µs 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.924300017Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T04:01:30.141 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.92431574Z level=info msg="Migration successfully executed" id="Update dashboard_snapshot table charset" duration=16.924µs 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.924704254Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.926580659Z level=info msg="Migration successfully executed" id="Add column external_delete_url to dashboard_snapshots table" duration=1.876138ms 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.926896349Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.928610256Z level=info msg="Migration successfully executed" id="Add encrypted dashboard json column" duration=1.713598ms 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.928938627Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.9289784Z level=info msg="Migration successfully executed" id="Change dashboard_encrypted column to MEDIUMBLOB" duration=40.274µs 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.929356225Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.929828824Z level=info msg="Migration successfully executed" id="create quota table v1" duration=472.335µs 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.930144298Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.930652932Z level=info msg="Migration successfully executed" id="create index UQE_quota_org_id_user_id_target - v1" duration=508.343µs 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.930966601Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.93098263Z level=info msg="Migration successfully executed" id="Update quota table charset" duration=17.469µs 2024-09-10T04:01:30.142 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.931350631Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.931863833Z level=info msg="Migration successfully executed" id="create plugin_setting table" duration=512.868µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.93217374Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.932734537Z level=info msg="Migration successfully executed" id="create index UQE_plugin_setting_org_id_plugin_id - v1" duration=560.613µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.933043971Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.934803708Z level=info msg="Migration successfully executed" id="Add column plugin_version to plugin_settings" duration=1.75955ms 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.935116083Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.935133116Z level=info msg="Migration successfully executed" id="Update plugin_setting table charset" duration=17.127µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.935524588Z level=info msg="Executing migration" id="create session table" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.936101865Z level=info msg="Migration successfully executed" id="create session table" duration=577.031µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.936446744Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.936530463Z level=info msg="Migration successfully executed" id="Drop old table playlist table" duration=83.906µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.936903282Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.936960571Z level=info msg="Migration successfully executed" id="Drop old table playlist_item table" duration=57.542µs 2024-09-10T04:01:30.143 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.937291114Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.937778951Z level=info msg="Migration successfully executed" id="create playlist table v2" duration=487.593µs 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.938108752Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.938601268Z level=info msg="Migration successfully executed" id="create playlist item table v2" duration=492.267µs 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.938918567Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.938934145Z level=info msg="Migration successfully executed" id="Update playlist table charset" duration=16.462µs 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.939308833Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.939324992Z level=info msg="Migration successfully executed" id="Update playlist_item table charset" duration=16.93µs 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.939710598Z level=info msg="Executing migration" id="Add playlist column created_at" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.941641917Z level=info msg="Migration successfully executed" id="Add playlist column created_at" duration=1.93127ms 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.941976218Z level=info msg="Executing migration" id="Add playlist column updated_at" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.943863535Z level=info msg="Migration successfully executed" id="Add playlist column updated_at" duration=1.887018ms 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.944167908Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T04:01:30.144 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.944228591Z level=info msg="Migration successfully executed" id="drop preferences table v2" duration=60.839µs 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.944598304Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.944657871Z level=info msg="Migration successfully executed" id="drop preferences table v3" duration=59.733µs 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.945034097Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.9455588Z level=info msg="Migration successfully executed" id="create preferences table v3" duration=525.067µs 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.945871093Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.94588704Z level=info msg="Migration successfully executed" id="Update preferences table charset" duration=16.733µs 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.946300697Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.948233284Z level=info msg="Migration successfully executed" id="Add column team_id in preferences" duration=1.932459ms 2024-09-10T04:01:30.145 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.948564185Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.948664999Z level=info msg="Migration successfully executed" id="Update team_id column values in preferences" duration=99.64µs 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.94903189Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.950972746Z level=info msg="Migration successfully executed" id="Add column week_start in preferences" duration=1.940694ms 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.951287899Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.953367299Z level=info msg="Migration successfully executed" id="Add column preferences.json_data" duration=2.079138ms 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.953717563Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.953759492Z level=info msg="Migration successfully executed" id="alter preferences.json_data to mediumtext v1" duration=42.461µs 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.954180404Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.954773184Z level=info msg="Migration successfully executed" id="Add preferences index org_id" duration=592.939µs 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.955096612Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.955649892Z level=info msg="Migration successfully executed" id="Add preferences index user_id" duration=553.2µs 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.955983989Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T04:01:30.146 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.956742176Z level=info msg="Migration successfully executed" id="create alert table v1" duration=757.923µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.957095688Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.957722564Z level=info msg="Migration successfully executed" id="add index alert org_id & id " duration=626.96µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.958063186Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.958607851Z level=info msg="Migration successfully executed" id="add index alert state" duration=544.567µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.958924182Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.959456926Z level=info msg="Migration successfully executed" id="add index alert dashboard_id" duration=532.576µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.959782707Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.960227987Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v1" duration=444.929µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.960552654Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.961116785Z level=info msg="Migration successfully executed" id="Add unique index alert_rule_tag.alert_id_tag_id" duration=563.896µs 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.961425224Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T04:01:30.147 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.961966498Z level=info msg="Migration successfully executed" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" duration=541.146µs 2024-09-10T04:01:30.148 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.962266382Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T04:01:30.148 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.968445384Z level=info msg="Migration successfully executed" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" duration=6.178003ms 2024-09-10T04:01:30.148 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.969203803Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T04:01:30.148 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.969678265Z level=info msg="Migration successfully executed" id="Create alert_rule_tag table v2" duration=474.322µs 2024-09-10T04:01:30.148 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.969986539Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.970526764Z level=info msg="Migration successfully executed" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" duration=539.948µs 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.970847423Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.971041771Z level=info msg="Migration successfully executed" id="copy alert_rule_tag v1 to v2" duration=194.386µs 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.971347275Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.971719313Z level=info msg="Migration successfully executed" id="drop table alert_rule_tag_v1" duration=372.906µs 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.972024393Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T04:01:30.150 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.972532897Z level=info msg="Migration successfully executed" id="create alert_notification table v1" duration=508.236µs 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.972840839Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.975013251Z level=info msg="Migration successfully executed" id="Add column is_default" duration=2.172153ms 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.975319846Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.977669314Z level=info msg="Migration successfully executed" id="Add column frequency" duration=2.349111ms 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.97797875Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.98049601Z level=info msg="Migration successfully executed" id="Add column send_reminder" duration=2.518747ms 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.980807291Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.983001971Z level=info msg="Migration successfully executed" id="Add column disable_resolve_message" duration=2.19436ms 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.983314199Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.983841676Z level=info msg="Migration successfully executed" id="add index alert_notification org_id & name" duration=527.408µs 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.984149436Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.984165541Z level=info msg="Migration successfully executed" id="Update alert table charset" duration=16.809µs 2024-09-10T04:01:30.151 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.984556668Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.984573767Z level=info msg="Migration successfully executed" id="Update alert_notification table charset" duration=17.767µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.984939139Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.985408501Z level=info msg="Migration successfully executed" id="create notification_journal table v1" duration=468.106µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.985757337Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.986288145Z level=info msg="Migration successfully executed" id="add index notification_journal org_id & alert_id & notifier_id" duration=529.561µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.986604992Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.987134477Z level=info msg="Migration successfully executed" id="drop alert_notification_journal" duration=529.344µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.987448762Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.987967836Z level=info msg="Migration successfully executed" id="create alert_notification_state table v1" duration=518.95µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.988274636Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.988811776Z level=info msg="Migration successfully executed" id="add index alert_notification_state org_id & alert_id & notifier_id" duration=536.753µs 2024-09-10T04:01:30.152 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.989113317Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.991375923Z level=info msg="Migration successfully executed" id="Add for to alert table" duration=2.262348ms 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.991692487Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.994003111Z level=info msg="Migration successfully executed" id="Add column uid in alert_notification" duration=2.311348ms 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.994310242Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.994426839Z level=info msg="Migration successfully executed" id="Update uid column values in alert_notification" duration=116.449µs 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.994811661Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.995331718Z level=info msg="Migration successfully executed" id="Add unique index alert_notification_org_id_uid" duration=519.881µs 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.995658371Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.996182247Z level=info msg="Migration successfully executed" id="Remove unique index org_id_name" duration=524.403µs 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.996488082Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.99882021Z level=info msg="Migration successfully executed" id="Add column secure_settings in alert_notification" duration=2.331823ms 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.999125735Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T04:01:30.153 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.999165771Z level=info msg="Migration successfully executed" id="alter alert.settings to mediumtext" duration=40.486µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:29.999590159Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.000111291Z level=info msg="Migration successfully executed" id="Add non-unique index alert_notification_state_alert_id" duration=520.976µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.000411065Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.001006456Z level=info msg="Migration successfully executed" id="Add non-unique index alert_rule_tag_alert_id" duration=595.09µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.001329307Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.001390149Z level=info msg="Migration successfully executed" id="Drop old annotation table v4" duration=60.977µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.001762112Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.002321798Z level=info msg="Migration successfully executed" id="create annotation table v5" duration=559.538µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.002646504Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.003171866Z level=info msg="Migration successfully executed" id="add index annotation 0 v3" duration=525.324µs 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.003521946Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T04:01:30.154 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.004162693Z level=info msg="Migration successfully executed" id="add index annotation 1 v3" duration=639.642µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.004562545Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.005107336Z level=info msg="Migration successfully executed" id="add index annotation 2 v3" duration=545.51µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.005416868Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.006016204Z level=info msg="Migration successfully executed" id="add index annotation 3 v3" duration=599.048µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.006338939Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.006979513Z level=info msg="Migration successfully executed" id="add index annotation 4 v3" duration=640.411µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.007315937Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.007332146Z level=info msg="Migration successfully executed" id="Update annotation table charset" duration=17.039µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.007710317Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.010318867Z level=info msg="Migration successfully executed" id="Add column region_id to annotation table" duration=2.608235ms 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.010668191Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.011205172Z level=info msg="Migration successfully executed" id="Drop category_id index" duration=537.059µs 2024-09-10T04:01:30.155 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.011514908Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.013944189Z level=info msg="Migration successfully executed" id="Add column tags to annotation table" duration=2.42898ms 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.014261478Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.014712341Z level=info msg="Migration successfully executed" id="Create annotation_tag table v2" duration=450.407µs 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.015022075Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.015581153Z level=info msg="Migration successfully executed" id="Add unique index annotation_tag.annotation_id_tag_id" duration=558.936µs 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.015900086Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.016523878Z level=info msg="Migration successfully executed" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" duration=622.748µs 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.01693741Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.02454787Z level=info msg="Migration successfully executed" id="Rename table annotation_tag to annotation_tag_v2 - v2" duration=7.609643ms 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.024915897Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.025386395Z level=info msg="Migration successfully executed" id="Create annotation_tag table v3" duration=468.331µs 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.025707576Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T04:01:30.156 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.026256989Z level=info msg="Migration successfully executed" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" duration=549.393µs 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.026595285Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.026791862Z level=info msg="Migration successfully executed" id="copy annotation_tag v2 to v3" duration=197.887µs 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.027104452Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.027481349Z level=info msg="Migration successfully executed" id="drop table annotation_tag_v2" duration=376.578µs 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.027782018Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.027894344Z level=info msg="Migration successfully executed" id="Update alert annotations and set TEXT to empty" duration=112.612µs 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.028259323Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.03073597Z level=info msg="Migration successfully executed" id="Add created time to annotation table" duration=2.477791ms 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.031066662Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.033525951Z level=info msg="Migration successfully executed" id="Add updated time to annotation table" duration=2.458964ms 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.033863437Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.034399282Z level=info msg="Migration successfully executed" id="Add index for created in annotation table" duration=535.678µs 2024-09-10T04:01:30.157 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.034721956Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.035252763Z level=info msg="Migration successfully executed" id="Add index for updated in annotation table" duration=530.491µs 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.035574862Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.035720078Z level=info msg="Migration successfully executed" id="Convert existing annotations from seconds to milliseconds" duration=145.276µs 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.036088068Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.038595782Z level=info msg="Migration successfully executed" id="Add epoch_end column" duration=2.50754ms 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.038926315Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.039475829Z level=info msg="Migration successfully executed" id="Add index for epoch_end" duration=549.232µs 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.039789344Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.039900856Z level=info msg="Migration successfully executed" id="Make epoch_end the same as epoch" duration=109.922µs 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.040263081Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.040489323Z level=info msg="Migration successfully executed" id="Move region to single row" duration=226.385µs 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.040854347Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T04:01:30.158 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.041400035Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch from annotation table" duration=546.642µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.041714686Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.042253747Z level=info msg="Migration successfully executed" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" duration=538.974µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.042578753Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.04313001Z level=info msg="Migration successfully executed" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" duration=551.629µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.043485299Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.044025924Z level=info msg="Migration successfully executed" id="Add index for org_id_epoch_end_epoch on annotation table" duration=567.91µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.044344729Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.044887075Z level=info msg="Migration successfully executed" id="Remove index org_id_epoch_epoch_end from annotation table" duration=542.32µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.045219729Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.045798973Z level=info msg="Migration successfully executed" id="Add index for alert_id on annotation table" duration=578.988µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.046135883Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.046180805Z level=info msg="Migration successfully executed" id="Increase tags column to length 4096" duration=44.135µs 2024-09-10T04:01:30.159 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.046595445Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.04711996Z level=info msg="Migration successfully executed" id="create test_data table" duration=524.225µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.047443567Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.047976928Z level=info msg="Migration successfully executed" id="create dashboard_version table v1" duration=533.208µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.048289112Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.048823798Z level=info msg="Migration successfully executed" id="add index dashboard_version.dashboard_id" duration=534.547µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.049136242Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.049720042Z level=info msg="Migration successfully executed" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" duration=584.764µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.050031368Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.050146639Z level=info msg="Migration successfully executed" id="Set dashboard version to 1 where 0" duration=115.498µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.050494867Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.050730384Z level=info msg="Migration successfully executed" id="save existing dashboard data in dashboard_version table v1" duration=234.165µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.051098371Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.051139485Z level=info msg="Migration successfully executed" id="alter dashboard_version.data to mediumtext v1" duration=41.371µs 2024-09-10T04:01:30.160 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.051484489Z level=info msg="Executing migration" id="create team table" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.051955407Z level=info msg="Migration successfully executed" id="create team table" duration=470.355µs 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.052263579Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.052891143Z level=info msg="Migration successfully executed" id="add index team.org_id" duration=626.207µs 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.053214912Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.053778979Z level=info msg="Migration successfully executed" id="add unique index team_org_id_name" duration=563.865µs 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.054087741Z level=info msg="Executing migration" id="Add column uid in team" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.05677802Z level=info msg="Migration successfully executed" id="Add column uid in team" duration=2.690021ms 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.05708318Z level=info msg="Executing migration" id="Update uid column values in team" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.057197795Z level=info msg="Migration successfully executed" id="Update uid column values in team" duration=114.673µs 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.057549955Z level=info msg="Executing migration" id="Add unique index team_org_id_uid" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.058109516Z level=info msg="Migration successfully executed" id="Add unique index team_org_id_uid" duration=559.319µs 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.058424725Z level=info msg="Executing migration" id="create team member table" 2024-09-10T04:01:30.161 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.058941922Z level=info msg="Migration successfully executed" id="create team member table" duration=516.952µs 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.059251742Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.059799883Z level=info msg="Migration successfully executed" id="add index team_member.org_id" duration=548.018µs 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.060781436Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.061422121Z level=info msg="Migration successfully executed" id="add unique index team_member_org_id_team_id_user_id" duration=642.613µs 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.061753223Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.062315503Z level=info msg="Migration successfully executed" id="add index team_member.team_id" duration=562.547µs 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.062662483Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.065630867Z level=info msg="Migration successfully executed" id="Add column email to team table" duration=2.968269ms 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.065943801Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.068977999Z level=info msg="Migration successfully executed" id="Add column external to team_member table" duration=3.034331ms 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.069291892Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.072116031Z level=info msg="Migration successfully executed" id="Add column permission to team_member table" duration=2.823762ms 2024-09-10T04:01:30.162 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.072433236Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.073074552Z level=info msg="Migration successfully executed" id="create dashboard acl table" duration=639.865µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.07341422Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.074020217Z level=info msg="Migration successfully executed" id="add index dashboard_acl_dashboard_id" duration=605.684µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.074339066Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.075005128Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_user_id" duration=665.717µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.07537004Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.075988175Z level=info msg="Migration successfully executed" id="add unique index dashboard_acl_dashboard_id_team_id" duration=616.851µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.07630665Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.076882933Z level=info msg="Migration successfully executed" id="add index dashboard_acl_user_id" duration=574.496µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.077199277Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.077788045Z level=info msg="Migration successfully executed" id="add index dashboard_acl_team_id" duration=588.49µs 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.078109465Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T04:01:30.163 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.078718624Z level=info msg="Migration successfully executed" id="add index dashboard_acl_org_id_role" duration=608.941µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.079036903Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.079657597Z level=info msg="Migration successfully executed" id="add index dashboard_permission" duration=620.212µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.079973602Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.08031836Z level=info msg="Migration successfully executed" id="save default acl rules in dashboard_acl table" duration=344.726µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.080701013Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.080847429Z level=info msg="Migration successfully executed" id="delete acl rules for deleted dashboards and folders" duration=146.58µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.081199172Z level=info msg="Executing migration" id="create tag table" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.081717626Z level=info msg="Migration successfully executed" id="create tag table" duration=518.321µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.082044109Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.082636057Z level=info msg="Migration successfully executed" id="add index tag.key_value" duration=591.787µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.082960284Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.083439136Z level=info msg="Migration successfully executed" id="create login attempt table" duration=478.607µs 2024-09-10T04:01:30.164 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.083765787Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.084351051Z level=info msg="Migration successfully executed" id="add index login_attempt.username" duration=585.015µs 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.084693404Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.085269971Z level=info msg="Migration successfully executed" id="drop index IDX_login_attempt_username - v1" duration=576.521µs 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.085583754Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.09431873Z level=info msg="Migration successfully executed" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" duration=8.734536ms 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.094752468Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.095225426Z level=info msg="Migration successfully executed" id="create login_attempt v2" duration=472.803µs 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.095538141Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.096109758Z level=info msg="Migration successfully executed" id="create index IDX_login_attempt_username - v2" duration=571.431µs 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.096428408Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.096651582Z level=info msg="Migration successfully executed" id="copy login_attempt v1 to v2" duration=223.11µs 2024-09-10T04:01:30.165 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.096991314Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.097404276Z level=info msg="Migration successfully executed" id="drop login_attempt_tmp_qwerty" duration=411.44µs 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.09772488Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.098195964Z level=info msg="Migration successfully executed" id="create user auth table" duration=471.849µs 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.098515156Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.099107223Z level=info msg="Migration successfully executed" id="create index IDX_user_auth_auth_module_auth_id - v1" duration=591.767µs 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.099420693Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.099480707Z level=info msg="Migration successfully executed" id="alter user_auth.auth_id to length 190" duration=60.827µs 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.099889303Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.103085458Z level=info msg="Migration successfully executed" id="Add OAuth access token to user_auth" duration=3.195685ms 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.103397886Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.106513553Z level=info msg="Migration successfully executed" id="Add OAuth refresh token to user_auth" duration=3.11539ms 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.106830231Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T04:01:30.166 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.109928649Z level=info msg="Migration successfully executed" id="Add OAuth token type to user_auth" duration=3.098933ms 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.110235832Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.113417193Z level=info msg="Migration successfully executed" id="Add OAuth expiry to user_auth" duration=3.181114ms 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.113748224Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.114309315Z level=info msg="Migration successfully executed" id="Add index to user_id column in user_auth" duration=560.881µs 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.114635035Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.117796089Z level=info msg="Migration successfully executed" id="Add OAuth ID token to user_auth" duration=3.159608ms 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.118106165Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.118625157Z level=info msg="Migration successfully executed" id="create server_lock table" duration=518.745µs 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.118952337Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.119523514Z level=info msg="Migration successfully executed" id="add index server_lock.operation_uid" duration=570.998µs 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.119843949Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.120397296Z level=info msg="Migration successfully executed" id="create user auth token table" duration=553.197µs 2024-09-10T04:01:30.167 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.120723211Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.12129304Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.auth_token" duration=569.675µs 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.12162114Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.12219427Z level=info msg="Migration successfully executed" id="add unique index user_auth_token.prev_auth_token" duration=573.815µs 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.122509929Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.123148774Z level=info msg="Migration successfully executed" id="add index user_auth_token.user_id" duration=638.747µs 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.123479334Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.126841952Z level=info msg="Migration successfully executed" id="Add revoked_at to the user auth token" duration=3.362409ms 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.127151327Z level=info msg="Executing migration" id="add index user_auth_token.revoked_at" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.127743105Z level=info msg="Migration successfully executed" id="add index user_auth_token.revoked_at" duration=591.498µs 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.128055667Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.12862294Z level=info msg="Migration successfully executed" id="create cache_data table" duration=567.09µs 2024-09-10T04:01:30.168 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.128953932Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.12954287Z level=info msg="Migration successfully executed" id="add unique index cache_data.cache_key" duration=588.818µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.129860165Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.130433551Z level=info msg="Migration successfully executed" id="create short_url table v1" duration=573.116µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.130784511Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.131392094Z level=info msg="Migration successfully executed" id="add index short_url.org_id-uid" duration=607.528µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.131719095Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.131759212Z level=info msg="Migration successfully executed" id="alter table short_url alter column created_by type to bigint" duration=40.502µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.132126859Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.132183635Z level=info msg="Migration successfully executed" id="delete alert_definition table" duration=56.928µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.13250991Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.13306203Z level=info msg="Migration successfully executed" id="recreate alert_definition table" duration=551.91µs 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.133374235Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T04:01:30.169 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.133990149Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and title columns" duration=615.726µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.134320981Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.134943244Z level=info msg="Migration successfully executed" id="add index in alert_definition on org_id and uid columns" duration=622.015µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.135273153Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.135314726Z level=info msg="Migration successfully executed" id="alter alert_definition table data column to mediumtext in mysql" duration=41.872µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.135700536Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.136290056Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and title columns" duration=589.368µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.136613566Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.137200031Z level=info msg="Migration successfully executed" id="drop index in alert_definition on org_id and uid columns" duration=586.463µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.137518575Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.138165111Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and title columns" duration=646.345µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.138719542Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.139340039Z level=info msg="Migration successfully executed" id="add unique index in alert_definition on org_id and uid columns" duration=620.307µs 2024-09-10T04:01:30.170 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.139661909Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T04:01:30.171 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.143339871Z level=info msg="Migration successfully executed" id="Add column paused in alert_definition" duration=3.677727ms 2024-09-10T04:01:30.171 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.143663468Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T04:01:30.396 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14436318Z level=info msg="Migration successfully executed" id="drop alert_definition table" duration=698.069µs 2024-09-10T04:01:30.396 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.144698312Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14476071Z level=info msg="Migration successfully executed" id="delete alert_definition_version table" duration=62.712µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14512826Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.145718236Z level=info msg="Migration successfully executed" id="recreate alert_definition_version table" duration=589.261µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.146030439Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.146680934Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_id and version columns" duration=650.139µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.146989843Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.147619095Z level=info msg="Migration successfully executed" id="add index in alert_definition_version table on alert_definition_uid and version columns" duration=628.914µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.147959567Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14799984Z level=info msg="Migration successfully executed" id="alter alert_definition_version table data column to mediumtext in mysql" duration=40.624µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.148364532Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14904999Z level=info msg="Migration successfully executed" id="drop alert_definition_version table" duration=683.563µs 2024-09-10T04:01:30.397 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.149369284Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.14998653Z level=info msg="Migration successfully executed" id="create alert_instance table" duration=617.337µs 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.150297363Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.150929481Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" duration=631.95µs 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.151234131Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.151850881Z level=info msg="Migration successfully executed" id="add index in alert_instance table on def_org_id, current_state columns" duration=616.575µs 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.152161188Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.155829055Z level=info msg="Migration successfully executed" id="add column current_state_end to alert_instance" duration=3.667567ms 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.156137173Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.156729537Z level=info msg="Migration successfully executed" id="remove index def_org_id, def_uid, current_state on alert_instance" duration=592.374µs 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.157031817Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.157618426Z level=info msg="Migration successfully executed" id="remove index def_org_id, current_state on alert_instance" duration=585.346µs 2024-09-10T04:01:30.398 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.157925445Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.177286505Z level=info msg="Migration successfully executed" id="rename def_org_id to rule_org_id in alert_instance" duration=19.360676ms 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.177705912Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.197652956Z level=info msg="Migration successfully executed" id="rename def_uid to rule_uid in alert_instance" duration=19.943463ms 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.198106252Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.198773899Z level=info msg="Migration successfully executed" id="add index rule_org_id, rule_uid, current_state on alert_instance" duration=667.534µs 2024-09-10T04:01:30.399 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.199087938Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T04:01:30.400 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.199749623Z level=info msg="Migration successfully executed" id="add index rule_org_id, current_state on alert_instance" duration=655.028µs 2024-09-10T04:01:30.400 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.200072365Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T04:01:30.400 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.203814138Z level=info msg="Migration successfully executed" id="add current_reason column related to current_state" duration=3.740387ms 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.204186734Z level=info msg="Executing migration" id="add result_fingerprint column to alert_instance" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.207701331Z level=info msg="Migration successfully executed" id="add result_fingerprint column to alert_instance" duration=3.51477ms 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.208049162Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.208728305Z level=info msg="Migration successfully executed" id="create alert_rule table" duration=679µs 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.209068879Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.209745414Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and title columns" duration=676.222µs 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.210079916Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.210741478Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id and uid columns" duration=661.326µs 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.211077616Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.211823169Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" duration=745.091µs 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.212207968Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T04:01:30.401 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.212259798Z level=info msg="Migration successfully executed" id="alter alert_rule table data column to mediumtext in mysql" duration=52.252µs 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.212661777Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.217051253Z level=info msg="Migration successfully executed" id="add column for to alert_rule" duration=4.387537ms 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.217509123Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.222636488Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule" duration=5.127053ms 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.223067953Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.228167372Z level=info msg="Migration successfully executed" id="add column labels to alert_rule" duration=5.099191ms 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.228593381Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.229400874Z level=info msg="Migration successfully executed" id="remove unique index from alert_rule on org_id, title columns" duration=807.27µs 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.229806351Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.230622395Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, namespase_uid and title columns" duration=815.574µs 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.231022532Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.235890105Z level=info msg="Migration successfully executed" id="add dashboard_uid column to alert_rule" duration=4.867452ms 2024-09-10T04:01:30.402 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.236256747Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.239890853Z level=info msg="Migration successfully executed" id="add panel_id column to alert_rule" duration=3.633971ms 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.240213566Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.240944842Z level=info msg="Migration successfully executed" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" duration=728.922µs 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.2412693Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.244915455Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule" duration=3.647179ms 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.245234457Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.248892765Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule table" duration=3.658069ms 2024-09-10T04:01:30.403 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.249217905Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T04:01:30.404 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.249257916Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule table" duration=40.382µs 2024-09-10T04:01:30.404 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.249667489Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T04:01:30.404 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.250429179Z level=info msg="Migration successfully executed" id="create alert_rule_version table" duration=761.954µs 2024-09-10T04:01:30.404 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.250789361Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T04:01:30.404 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.251494501Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" duration=705.294µs 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.251815966Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.252557537Z level=info msg="Migration successfully executed" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" duration=741.156µs 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.252898466Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.252938899Z level=info msg="Migration successfully executed" id="alter alert_rule_version table data column to mediumtext in mysql" duration=40.766µs 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.253333796Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.257199655Z level=info msg="Migration successfully executed" id="add column for to alert_rule_version" duration=3.865585ms 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.257530324Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.261355873Z level=info msg="Migration successfully executed" id="add column annotations to alert_rule_version" duration=3.825309ms 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.261686411Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.265490629Z level=info msg="Migration successfully executed" id="add column labels to alert_rule_version" duration=3.804072ms 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.26580853Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.269569987Z level=info msg="Migration successfully executed" id="add rule_group_idx column to alert_rule_version" duration=3.761349ms 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.269884689Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T04:01:30.405 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.27367889Z level=info msg="Migration successfully executed" id="add is_paused column to alert_rule_versions table" duration=3.793863ms 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.274021291Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.274060773Z level=info msg="Migration successfully executed" id="fix is_paused column for alert_rule_version table" duration=41.56µs 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.274472752Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.274976226Z level=info msg="Migration successfully executed" id=create_alert_configuration_table duration=517.552µs 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.27529596Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.279197254Z level=info msg="Migration successfully executed" id="Add column default in alert_configuration" duration=3.901457ms 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.280610929Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.280656707Z level=info msg="Migration successfully executed" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" duration=46.828µs 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.281063968Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.284977692Z level=info msg="Migration successfully executed" id="add column org_id in alert_configuration" duration=3.913837ms 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.285292226Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.285972048Z level=info msg="Migration successfully executed" id="add index in alert_configuration table on org_id column" duration=680µs 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.286295782Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T04:01:30.406 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.290225726Z level=info msg="Migration successfully executed" id="add configuration_hash column to alert_configuration" duration=3.929588ms 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.290554608Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.2911749Z level=info msg="Migration successfully executed" id=create_ngalert_configuration_table duration=619.875µs 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.291509664Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.292138098Z level=info msg="Migration successfully executed" id="add index in ngalert_configuration on org_id column" duration=628.226µs 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.292466333Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T04:01:30.407 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.296428968Z level=info msg="Migration successfully executed" id="add column send_alerts_to in ngalert_configuration" duration=3.970289ms 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.296806296Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.297301822Z level=info msg="Migration successfully executed" id="create provenance_type table" duration=495.422µs 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.297645516Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.298271133Z level=info msg="Migration successfully executed" id="add index to uniquify (record_key, record_type, org_id) columns" duration=625.484µs 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.298602874Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.299116157Z level=info msg="Migration successfully executed" id="create alert_image table" duration=512.426µs 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.299436118Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.300049224Z level=info msg="Migration successfully executed" id="add unique index on token to alert_image table" duration=613.033µs 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.300376358Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T04:01:30.408 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.30041598Z level=info msg="Migration successfully executed" id="support longer URLs in alert_image table" duration=39.951µs 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.300799304Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.301405094Z level=info msg="Migration successfully executed" id=create_alert_configuration_history_table duration=605.524µs 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.301748104Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.30235907Z level=info msg="Migration successfully executed" id="drop non-unique orgID index on alert_configuration" duration=609.381µs 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.302675494Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.302907014Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.303252609Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.303537671Z level=info msg="Migration successfully executed" id="extract alertmanager configuration history to separate table" duration=284.88µs 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.303837594Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.304444351Z level=info msg="Migration successfully executed" id="add unique index on orgID to alert_configuration" duration=606.548µs 2024-09-10T04:01:30.409 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.304756561Z level=info msg="Executing migration" id="add last_applied column to alert_configuration_history" 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.308765765Z level=info msg="Migration successfully executed" id="add last_applied column to alert_configuration_history" duration=4.00885ms 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.309085856Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.309809718Z level=info msg="Migration successfully executed" id="create library_element table v1" duration=723.543µs 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.31014723Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.310818794Z level=info msg="Migration successfully executed" id="add index library_element org_id-folder_id-name-kind" duration=671.249µs 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.311137335Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.31168935Z level=info msg="Migration successfully executed" id="create library_element_connection table v1" duration=551.727µs 2024-09-10T04:01:30.410 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.312009017Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.312675369Z level=info msg="Migration successfully executed" id="add index library_element_connection element_id-kind-connection_id" duration=665.106µs 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.312997957Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.313753454Z level=info msg="Migration successfully executed" id="add unique index library_element org_id_uid" duration=757.502µs 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.314097433Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.314113852Z level=info msg="Migration successfully executed" id="increase max description length to 2048" duration=17.297µs 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.314500282Z level=info msg="Executing migration" id="alter library_element model to mediumtext" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.31454692Z level=info msg="Migration successfully executed" id="alter library_element model to mediumtext" duration=47.261µs 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.31493378Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.315129889Z level=info msg="Migration successfully executed" id="clone move dashboard alerts to unified alerting" duration=196.519µs 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.31547554Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T04:01:30.411 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.316183369Z level=info msg="Migration successfully executed" id="create data_keys table" duration=708.171µs 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.316537435Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.317094872Z level=info msg="Migration successfully executed" id="create secrets table" duration=557.894µs 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.317418244Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.341616265Z level=info msg="Migration successfully executed" id="rename data_keys name column to id" duration=24.192263ms 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.342082337Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.346813313Z level=info msg="Migration successfully executed" id="add name column into data_keys" duration=4.730726ms 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.347181047Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.347283959Z level=info msg="Migration successfully executed" id="copy data_keys id column values into name" duration=102.952µs 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.347664701Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.371202509Z level=info msg="Migration successfully executed" id="rename data_keys name column to label" duration=23.52725ms 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.371628592Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.39546673Z level=info msg="Migration successfully executed" id="rename data_keys id column back to name" duration=23.821593ms 2024-09-10T04:01:30.412 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.395928226Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.396669427Z level=info msg="Migration successfully executed" id="create kv_store table v1" duration=741.047µs 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.397044615Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.397766883Z level=info msg="Migration successfully executed" id="add index kv_store.org_id-namespace-key" duration=722.029µs 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.398114509Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.398275219Z level=info msg="Migration successfully executed" id="update dashboard_uid and panel_id from existing annotations" duration=160.751µs 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.398689086Z level=info msg="Executing migration" id="create permission table" 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.399347608Z level=info msg="Migration successfully executed" id="create permission table" duration=658.459µs 2024-09-10T04:01:30.413 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.399717922Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.400359195Z level=info msg="Migration successfully executed" id="add unique index permission.role_id" duration=641.118µs 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.400716765Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.401394899Z level=info msg="Migration successfully executed" id="add unique index role_id_action_scope" duration=677.846µs 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.401760433Z level=info msg="Executing migration" id="create role table" 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.402370944Z level=info msg="Migration successfully executed" id="create role table" duration=610.151µs 2024-09-10T04:01:30.662 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.402728952Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.407660948Z level=info msg="Migration successfully executed" id="add column display_name" duration=4.931248ms 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.40802592Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.412592945Z level=info msg="Migration successfully executed" id="add column group_name" duration=4.568668ms 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.41294348Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.413634275Z level=info msg="Migration successfully executed" id="add index role.org_id" duration=690.687µs 2024-09-10T04:01:30.663 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.413970462Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.414676426Z level=info msg="Migration successfully executed" id="add unique index role_org_id_name" duration=705.652µs 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.415006398Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.415701076Z level=info msg="Migration successfully executed" id="add index role_org_id_uid" duration=694.619µs 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.416371152Z level=info msg="Executing migration" id="create team role table" 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.416956951Z level=info msg="Migration successfully executed" id="create team role table" duration=585.465µs 2024-09-10T04:01:30.664 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.41730453Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.418011845Z level=info msg="Migration successfully executed" id="add index team_role.org_id" duration=707.04µs 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.418350929Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.41908083Z level=info msg="Migration successfully executed" id="add unique index team_role_org_id_team_id_role_id" duration=729.467µs 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.419427554Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.420092997Z level=info msg="Migration successfully executed" id="add index team_role.team_id" duration=663.563µs 2024-09-10T04:01:30.665 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.420430838Z level=info msg="Executing migration" id="create user role table" 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.421014969Z level=info msg="Migration successfully executed" id="create user role table" duration=583.837µs 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.421510459Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.422173111Z level=info msg="Migration successfully executed" id="add index user_role.org_id" duration=662.268µs 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.422512659Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.423171036Z level=info msg="Migration successfully executed" id="add unique index user_role_org_id_user_id_role_id" duration=658.087µs 2024-09-10T04:01:30.666 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.423532348Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T04:01:30.667 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.424176304Z level=info msg="Migration successfully executed" id="add index user_role.user_id" duration=643.847µs 2024-09-10T04:01:30.667 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.424517836Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T04:01:30.667 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.425075702Z level=info msg="Migration successfully executed" id="create builtin role table" duration=557.535µs 2024-09-10T04:01:30.667 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.425406882Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T04:01:30.667 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.426088038Z level=info msg="Migration successfully executed" id="add index builtin_role.role_id" duration=682.096µs 2024-09-10T04:01:30.668 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.426413253Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T04:01:30.671 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.427094636Z level=info msg="Migration successfully executed" id="add index builtin_role.name" duration=681.043µs 2024-09-10T04:01:30.671 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.427420869Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T04:01:30.671 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.432526753Z level=info msg="Migration successfully executed" id="Add column org_id to builtin_role table" duration=5.1037ms 2024-09-10T04:01:30.671 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.432856087Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T04:01:30.671 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.43353305Z level=info msg="Migration successfully executed" id="add index builtin_role.org_id" duration=676.619µs 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.433865668Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.434550045Z level=info msg="Migration successfully executed" id="add unique index builtin_role_org_id_role_id_role" duration=684.253µs 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.434879437Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.435562824Z level=info msg="Migration successfully executed" id="Remove unique index role_org_id_uid" duration=683.264µs 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.435882523Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T04:01:30.672 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.436548443Z level=info msg="Migration successfully executed" id="add unique index role.uid" duration=665.777µs 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.436876458Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.437391316Z level=info msg="Migration successfully executed" id="create seed assignment table" duration=515.122µs 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.437737397Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.438421376Z level=info msg="Migration successfully executed" id="add unique index builtin_role_role_name" duration=683.847µs 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.438760842Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T04:01:30.673 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.443859605Z level=info msg="Migration successfully executed" id="add column hidden to role table" duration=5.098489ms 2024-09-10T04:01:30.674 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.444198295Z level=info msg="Executing migration" id="permission kind migration" 2024-09-10T04:01:30.674 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.449218287Z level=info msg="Migration successfully executed" id="permission kind migration" duration=5.043937ms 2024-09-10T04:01:30.674 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.449550316Z level=info msg="Executing migration" id="permission attribute migration" 2024-09-10T04:01:30.674 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.45443029Z level=info msg="Migration successfully executed" id="permission attribute migration" duration=4.879389ms 2024-09-10T04:01:30.674 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.454781036Z level=info msg="Executing migration" id="permission identifier migration" 2024-09-10T04:01:30.675 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.460060876Z level=info msg="Migration successfully executed" id="permission identifier migration" duration=5.278012ms 2024-09-10T04:01:30.675 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.460436603Z level=info msg="Executing migration" id="add permission identifier index" 2024-09-10T04:01:30.675 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46127131Z level=info msg="Migration successfully executed" id="add permission identifier index" duration=834.13µs 2024-09-10T04:01:30.675 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46170877Z level=info msg="Executing migration" id="add permission action scope role_id index" 2024-09-10T04:01:30.675 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.462507562Z level=info msg="Migration successfully executed" id="add permission action scope role_id index" duration=800.421µs 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.462905312Z level=info msg="Executing migration" id="remove permission role_id action scope index" 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46361501Z level=info msg="Migration successfully executed" id="remove permission role_id action scope index" duration=709.645µs 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.463955065Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.464571256Z level=info msg="Migration successfully executed" id="create query_history table v1" duration=615.895µs 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.464894342Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T04:01:30.676 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.465576797Z level=info msg="Migration successfully executed" id="add index query_history.org_id-created_by-datasource_uid" duration=682.119µs 2024-09-10T04:01:30.677 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.465901815Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T04:01:30.677 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.465953458Z level=info msg="Migration successfully executed" id="alter table query_history alter column created_by type to bigint" duration=51.807µs 2024-09-10T04:01:30.677 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.466328136Z level=info msg="Executing migration" id="rbac disabled migrator" 2024-09-10T04:01:30.678 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.466354217Z level=info msg="Migration successfully executed" id="rbac disabled migrator" duration=26.246µs 2024-09-10T04:01:30.678 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.466738883Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T04:01:30.678 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46701117Z level=info msg="Migration successfully executed" id="teams permissions migration" duration=272.305µs 2024-09-10T04:01:30.678 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.467371221Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T04:01:30.679 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.467736501Z level=info msg="Migration successfully executed" id="dashboard permissions" duration=365.784µs 2024-09-10T04:01:30.679 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.468113766Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T04:01:30.679 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.468551232Z level=info msg="Migration successfully executed" id="dashboard permissions uid scopes" duration=437.194µs 2024-09-10T04:01:30.679 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46898885Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T04:01:30.679 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.46914611Z level=info msg="Migration successfully executed" id="drop managed folder create actions" duration=157.649µs 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.469517739Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.469823791Z level=info msg="Migration successfully executed" id="alerting notification permissions" duration=305.924µs 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.470187329Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.470714146Z level=info msg="Migration successfully executed" id="create query_history_star table v1" duration=526.534µs 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.471038461Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.471721077Z level=info msg="Migration successfully executed" id="add index query_history.user_id-query_uid" duration=682.279µs 2024-09-10T04:01:30.680 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.47203998Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.477073636Z level=info msg="Migration successfully executed" id="add column org_id in query_history_star" duration=5.034504ms 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.477384205Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.477424528Z level=info msg="Migration successfully executed" id="alter table query_history_star_mig column user_id type to bigint" duration=40.723µs 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.477842567Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.478553905Z level=info msg="Migration successfully executed" id="create correlation table v1" duration=711.776µs 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.478897979Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.479584891Z level=info msg="Migration successfully executed" id="add index correlations.uid" duration=686.695µs 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.479906336Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.480605111Z level=info msg="Migration successfully executed" id="add index correlations.source_uid" duration=698.602µs 2024-09-10T04:01:30.681 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.480924889Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T04:01:30.682 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.486226582Z level=info msg="Migration successfully executed" id="add correlation config column" duration=5.300652ms 2024-09-10T04:01:30.682 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.486621829Z level=info msg="Executing migration" id="drop index IDX_correlation_uid - v1" 2024-09-10T04:01:30.682 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.487327702Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_uid - v1" duration=705.93µs 2024-09-10T04:01:30.682 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.487655223Z level=info msg="Executing migration" id="drop index IDX_correlation_source_uid - v1" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.488333127Z level=info msg="Migration successfully executed" id="drop index IDX_correlation_source_uid - v1" duration=677.773µs 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.48876684Z level=info msg="Executing migration" id="Rename table correlation to correlation_tmp_qwerty - v1" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.503633016Z level=info msg="Migration successfully executed" id="Rename table correlation to correlation_tmp_qwerty - v1" duration=14.856832ms 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.504177902Z level=info msg="Executing migration" id="create correlation v2" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.504997382Z level=info msg="Migration successfully executed" id="create correlation v2" duration=820.509µs 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.505332064Z level=info msg="Executing migration" id="create index IDX_correlation_uid - v2" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.506021459Z level=info msg="Migration successfully executed" id="create index IDX_correlation_uid - v2" duration=689.235µs 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.506355459Z level=info msg="Executing migration" id="create index IDX_correlation_source_uid - v2" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.507101292Z level=info msg="Migration successfully executed" id="create index IDX_correlation_source_uid - v2" duration=745.636µs 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.507439481Z level=info msg="Executing migration" id="create index IDX_correlation_org_id - v2" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.508124174Z level=info msg="Migration successfully executed" id="create index IDX_correlation_org_id - v2" duration=684.608µs 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.508443958Z level=info msg="Executing migration" id="copy correlation v1 to v2" 2024-09-10T04:01:30.683 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.50862411Z level=info msg="Migration successfully executed" id="copy correlation v1 to v2" duration=180.138µs 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.50900352Z level=info msg="Executing migration" id="drop correlation_tmp_qwerty" 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.509592383Z level=info msg="Migration successfully executed" id="drop correlation_tmp_qwerty" duration=588.642µs 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.509910452Z level=info msg="Executing migration" id="add provisioning column" 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.515120893Z level=info msg="Migration successfully executed" id="add provisioning column" duration=5.210123ms 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.515435456Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.515973435Z level=info msg="Migration successfully executed" id="create entity_events table" duration=537.711µs 2024-09-10T04:01:30.684 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.516292775Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.517011092Z level=info msg="Migration successfully executed" id="create dashboard public config v1" duration=717.976µs 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.517339182Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.517619623Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.517986206Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.518262525Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.518640556Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.519211576Z level=info msg="Migration successfully executed" id="Drop old dashboard public config table" duration=571.178µs 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.51953836Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.520208753Z level=info msg="Migration successfully executed" id="recreate dashboard public config v1" duration=670.212µs 2024-09-10T04:01:30.685 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.520567038Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.521302094Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v1" duration=733.674µs 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.521669811Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.522395584Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" duration=725.942µs 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.522728316Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.523417089Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_config_uid - v2" duration=688.714µs 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.523742564Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.524426259Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=683.309µs 2024-09-10T04:01:30.686 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.524861559Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.525662707Z level=info msg="Migration successfully executed" id="Drop public config table" duration=801.638µs 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.525990587Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.526715891Z level=info msg="Migration successfully executed" id="Recreate dashboard public config v2" duration=722.678µs 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.52703126Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.527721748Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_uid - v2" duration=688.532µs 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.528158678Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.5289018Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" duration=742.955µs 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.529243513Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.529967489Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_config_access_token - v2" duration=723.767µs 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.530309762Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.548551551Z level=info msg="Migration successfully executed" id="Rename table dashboard_public_config to dashboard_public - v2" duration=18.238481ms 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.549057553Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T04:01:30.687 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.554673855Z level=info msg="Migration successfully executed" id="add annotations_enabled column" duration=5.616749ms 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.555032058Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.560256039Z level=info msg="Migration successfully executed" id="add time_selection_enabled column" duration=5.223684ms 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.560605055Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.560784439Z level=info msg="Migration successfully executed" id="delete orphaned public dashboards" duration=177.456µs 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.561162972Z level=info msg="Executing migration" id="add share column" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.56641692Z level=info msg="Migration successfully executed" id="add share column" duration=5.253506ms 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.566848719Z level=info msg="Executing migration" id="backfill empty share column fields with default of public" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.567009076Z level=info msg="Migration successfully executed" id="backfill empty share column fields with default of public" duration=160.368µs 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.567395338Z level=info msg="Executing migration" id="create file table" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.568051754Z level=info msg="Migration successfully executed" id="create file table" duration=654.224µs 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.568402523Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.569107517Z level=info msg="Migration successfully executed" id="file table idx: path natural pk" duration=704.718µs 2024-09-10T04:01:30.688 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.56945094Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.570152007Z level=info msg="Migration successfully executed" id="file table idx: parent_folder_path_hash fast folder retrieval" duration=700.815µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.570507482Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.571064618Z level=info msg="Migration successfully executed" id="create file_meta table" duration=556.918µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.571410172Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.572122514Z level=info msg="Migration successfully executed" id="file table idx: path key" duration=712.131µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.572494604Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.572542893Z level=info msg="Migration successfully executed" id="set path collation in file table" duration=48.119µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.572957071Z level=info msg="Executing migration" id="migrate contents column to mediumblob for MySQL" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.57300656Z level=info msg="Migration successfully executed" id="migrate contents column to mediumblob for MySQL" duration=49.163µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.573514669Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.573901207Z level=info msg="Migration successfully executed" id="managed permissions migration" duration=387.138µs 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.5742636Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T04:01:30.689 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.574388241Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions migration" duration=124.817µs 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.574761325Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.57560154Z level=info msg="Migration successfully executed" id="RBAC action name migrator" duration=840.173µs 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.575962356Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.581267433Z level=info msg="Migration successfully executed" id="Add UID column to playlist" duration=5.304543ms 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.581625402Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.581757274Z level=info msg="Migration successfully executed" id="Update uid column values in playlist" duration=131.836µs 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.582141195Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.582973376Z level=info msg="Migration successfully executed" id="Add index for uid in playlist" duration=831.918µs 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.583334237Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.583621229Z level=info msg="Migration successfully executed" id="update group index for alert rules" duration=287.644µs 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.584006392Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T04:01:30.690 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.584160051Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated migration" duration=153.73µs 2024-09-10T04:01:30.691 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.584517988Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T04:01:30.691 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.584843926Z level=info msg="Migration successfully executed" id="admin only folder/dashboard permission" duration=325.744µs 2024-09-10T04:01:30.691 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.585231713Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T04:01:30.691 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.590622318Z level=info msg="Migration successfully executed" id="add action column to seed_assignment" duration=5.388469ms 2024-09-10T04:01:30.691 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.591024753Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.596427368Z level=info msg="Migration successfully executed" id="add scope column to seed_assignment" duration=5.401529ms 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.59683944Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.597594297Z level=info msg="Migration successfully executed" id="remove unique index builtin_role_role_name before nullable update" duration=754.941µs 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.597939896Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.660629751Z level=info msg="Migration successfully executed" id="update seed_assignment role_name column to nullable" duration=62.686481ms 2024-09-10T04:01:30.692 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.661198317Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T04:01:30.693 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.662046417Z level=info msg="Migration successfully executed" id="add unique index builtin_role_name back" duration=848.275µs 2024-09-10T04:01:30.693 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.662422371Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T04:01:30.693 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.663213208Z level=info msg="Migration successfully executed" id="add unique index builtin_role_action_scope" duration=792.34µs 2024-09-10T04:01:30.693 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.663572337Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T04:01:30.815 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T04:01:30.915 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:30.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:30.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:30.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:30.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: Deploying daemon node-exporter.a on smithi038 2024-09-10T04:01:30.916 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:30.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.681941531Z level=info msg="Migration successfully executed" id="add primary key to seed_assigment" duration=18.366228ms 2024-09-10T04:01:30.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.682438012Z level=info msg="Executing migration" id="add origin column to seed_assignment" 2024-09-10T04:01:30.917 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.688065683Z level=info msg="Migration successfully executed" id="add origin column to seed_assignment" duration=5.626015ms 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.688497631Z level=info msg="Executing migration" id="add origin to plugin seed_assignment" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.688721519Z level=info msg="Migration successfully executed" id="add origin to plugin seed_assignment" duration=224.259µs 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.689116257Z level=info msg="Executing migration" id="prevent seeding OnCall access" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.689234062Z level=info msg="Migration successfully executed" id="prevent seeding OnCall access" duration=117.917µs 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.689603401Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.689742643Z level=info msg="Migration successfully executed" id="managed folder permissions alert actions repeated fixed migration" duration=139.107µs 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.690105055Z level=info msg="Executing migration" id="managed folder permissions library panel actions migration" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.690229061Z level=info msg="Migration successfully executed" id="managed folder permissions library panel actions migration" duration=124.038µs 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.69060528Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.690751733Z level=info msg="Migration successfully executed" id="migrate external alertmanagers to datsourcse" duration=146.744µs 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.691138136Z level=info msg="Executing migration" id="create folder table" 2024-09-10T04:01:30.918 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.691817519Z level=info msg="Migration successfully executed" id="create folder table" duration=678.977µs 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.692301132Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.69315808Z level=info msg="Migration successfully executed" id="Add index for parent_uid" duration=856.722µs 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.693498198Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.694218513Z level=info msg="Migration successfully executed" id="Add unique index for folder.uid and folder.org_id" duration=720.067µs 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.694540005Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.69455696Z level=info msg="Migration successfully executed" id="Update folder title length" duration=19.665µs 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.694926064Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.695670178Z level=info msg="Migration successfully executed" id="Add unique index for folder.title and folder.parent_uid" duration=743.893µs 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.695990116Z level=info msg="Executing migration" id="Remove unique index for folder.title and folder.parent_uid" 2024-09-10T04:01:30.919 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.696700718Z level=info msg="Migration successfully executed" id="Remove unique index for folder.title and folder.parent_uid" duration=710.454µs 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.697006401Z level=info msg="Executing migration" id="Add unique index for title, parent_uid, and org_id" 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.697750884Z level=info msg="Migration successfully executed" id="Add unique index for title, parent_uid, and org_id" duration=744.377µs 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.698092234Z level=info msg="Executing migration" id="Sync dashboard and folder table" 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.698396997Z level=info msg="Migration successfully executed" id="Sync dashboard and folder table" duration=305.881µs 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.698701604Z level=info msg="Executing migration" id="Remove ghost folders from the folder table" 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.698887788Z level=info msg="Migration successfully executed" id="Remove ghost folders from the folder table" duration=186.095µs 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.699235789Z level=info msg="Executing migration" id="Remove unique index UQE_folder_uid_org_id" 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.69992384Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_uid_org_id" duration=687.96µs 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.700227707Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_uid" 2024-09-10T04:01:30.920 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.700975857Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_uid" duration=747.917µs 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.701278248Z level=info msg="Executing migration" id="Remove unique index UQE_folder_title_parent_uid_org_id" 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.701966849Z level=info msg="Migration successfully executed" id="Remove unique index UQE_folder_title_parent_uid_org_id" duration=688.486µs 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.702272361Z level=info msg="Executing migration" id="Add unique index UQE_folder_org_id_parent_uid_title" 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.703024648Z level=info msg="Migration successfully executed" id="Add unique index UQE_folder_org_id_parent_uid_title" duration=750.372µs 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.703327776Z level=info msg="Executing migration" id="Remove index IDX_folder_parent_uid_org_id" 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.704008322Z level=info msg="Migration successfully executed" id="Remove index IDX_folder_parent_uid_org_id" duration=680.427µs 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.704321455Z level=info msg="Executing migration" id="create anon_device table" 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.70490814Z level=info msg="Migration successfully executed" id="create anon_device table" duration=586.204µs 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.705240525Z level=info msg="Executing migration" id="add unique index anon_device.device_id" 2024-09-10T04:01:30.921 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.706056026Z level=info msg="Migration successfully executed" id="add unique index anon_device.device_id" duration=815.289µs 2024-09-10T04:01:30.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.706408357Z level=info msg="Executing migration" id="add index anon_device.updated_at" 2024-09-10T04:01:30.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.707139049Z level=info msg="Migration successfully executed" id="add index anon_device.updated_at" duration=731.083µs 2024-09-10T04:01:30.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.707479671Z level=info msg="Executing migration" id="create signing_key table" 2024-09-10T04:01:30.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.708123658Z level=info msg="Migration successfully executed" id="create signing_key table" duration=643.773µs 2024-09-10T04:01:30.922 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.708503943Z level=info msg="Executing migration" id="add unique index signing_key.key_id" 2024-09-10T04:01:30.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.709222232Z level=info msg="Migration successfully executed" id="add unique index signing_key.key_id" duration=717.978µs 2024-09-10T04:01:30.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.709575168Z level=info msg="Executing migration" id="set legacy alert migration status in kvstore" 2024-09-10T04:01:30.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.710309686Z level=info msg="Migration successfully executed" id="set legacy alert migration status in kvstore" duration=734.554µs 2024-09-10T04:01:30.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.710654858Z level=info msg="Executing migration" id="migrate record of created folders during legacy migration to kvstore" 2024-09-10T04:01:30.923 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.710858063Z level=info msg="Migration successfully executed" id="migrate record of created folders during legacy migration to kvstore" duration=203.793µs 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.711237037Z level=info msg="Executing migration" id="Add folder_uid for dashboard" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.7176958Z level=info msg="Migration successfully executed" id="Add folder_uid for dashboard" duration=6.456387ms 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.71811382Z level=info msg="Executing migration" id="Populate dashboard folder_uid column" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.718656928Z level=info msg="Migration successfully executed" id="Populate dashboard folder_uid column" duration=544.182µs 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.719073784Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.719848315Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title" duration=774.723µs 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.72018772Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_id_title" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.720931513Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_id_title" duration=743.686µs 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.721253774Z level=info msg="Executing migration" id="Delete unique index for dashboard_org_id_folder_uid_title" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.722016451Z level=info msg="Migration successfully executed" id="Delete unique index for dashboard_org_id_folder_uid_title" duration=762.619µs 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.722335106Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" 2024-09-10T04:01:30.924 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.723090805Z level=info msg="Migration successfully executed" id="Add unique index for dashboard_org_id_folder_uid_title_is_folder" duration=755.509µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.72340929Z level=info msg="Executing migration" id="Restore index for dashboard_org_id_folder_id_title" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.724124034Z level=info msg="Migration successfully executed" id="Restore index for dashboard_org_id_folder_id_title" duration=712.63µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.724433912Z level=info msg="Executing migration" id="create sso_setting table" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.725161447Z level=info msg="Migration successfully executed" id="create sso_setting table" duration=727.293µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.725534845Z level=info msg="Executing migration" id="copy kvstore migration status to each org" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.726033574Z level=info msg="Migration successfully executed" id="copy kvstore migration status to each org" duration=499.364µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.726345369Z level=info msg="Executing migration" id="add back entry for orgid=0 migrated status" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.72651731Z level=info msg="Migration successfully executed" id="add back entry for orgid=0 migrated status" duration=172.75µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.726891675Z level=info msg="Executing migration" id="alter kv_store.value to longtext" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.726933431Z level=info msg="Migration successfully executed" id="alter kv_store.value to longtext" duration=40.307µs 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.727289824Z level=info msg="Executing migration" id="add notification_settings column to alert_rule table" 2024-09-10T04:01:30.925 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.732949767Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule table" duration=5.658045ms 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.733333687Z level=info msg="Executing migration" id="add notification_settings column to alert_rule_version table" 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.739204105Z level=info msg="Migration successfully executed" id="add notification_settings column to alert_rule_version table" duration=5.868409ms 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.739691861Z level=info msg="Executing migration" id="removing scope from alert.instances:read action migration" 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.739944616Z level=info msg="Migration successfully executed" id="removing scope from alert.instances:read action migration" duration=255.993µs 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=migrator t=2024-09-10T04:01:30.740297795Z level=info msg="migrations completed" performed=547 skipped=0 duration=972.795094ms 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore t=2024-09-10T04:01:30.741193617Z level=info msg="Created default organization" 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=secrets t=2024-09-10T04:01:30.741716322Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T04:01:30.926 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=plugin.store t=2024-09-10T04:01:30.756368107Z level=info msg="Loading plugins..." 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=local.finder t=2024-09-10T04:01:30.805029277Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=plugin.store t=2024-09-10T04:01:30.805048488Z level=info msg="Plugins loaded" count=55 duration=48.680926ms 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=query_data t=2024-09-10T04:01:30.807083943Z level=info msg="Query Service initialization" 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=live.push_http t=2024-09-10T04:01:30.809250472Z level=info msg="Live Push Gateway initialization" 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.migration t=2024-09-10T04:01:30.81026235Z level=info msg=Starting 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.migration t=2024-09-10T04:01:30.810538752Z level=info msg="Applying transition" currentType=Legacy desiredType=UnifiedAlerting cleanOnDowngrade=false cleanOnUpgrade=false 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.migration orgID=1 t=2024-09-10T04:01:30.810836236Z level=info msg="Migrating alerts for organisation" 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.migration orgID=1 t=2024-09-10T04:01:30.811304206Z level=info msg="Alerts found to migrate" alerts=0 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.migration t=2024-09-10T04:01:30.812544735Z level=info msg="Completed alerting migration" 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.state.manager t=2024-09-10T04:01:30.826990466Z level=info msg="Running in alternative execution of Error/NoData mode" 2024-09-10T04:01:30.927 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=infra.usagestats.collector t=2024-09-10T04:01:30.828929802Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.datasources t=2024-09-10T04:01:30.830665749Z level=info msg="inserting datasource from configuration" name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.datasources t=2024-09-10T04:01:30.84335114Z level=info msg="inserting datasource from configuration" name=Loki uid=P8E80F9AEF21F6940 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.alerting t=2024-09-10T04:01:30.856498342Z level=info msg="starting to provision alerting" 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.alerting t=2024-09-10T04:01:30.85650973Z level=info msg="finished to provision alerting" 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=grafanaStorageLogger t=2024-09-10T04:01:30.856610909Z level=info msg="Storage starting" 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.state.manager t=2024-09-10T04:01:30.856904516Z level=info msg="Warming state cache for startup" 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.state.manager t=2024-09-10T04:01:30.857104952Z level=info msg="State cache has been initialized" states=0 duration=199.932µs 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.multiorg.alertmanager t=2024-09-10T04:01:30.857270201Z level=info msg="Starting MultiOrg Alertmanager" 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ngalert.scheduler t=2024-09-10T04:01:30.857295166Z level=info msg="Starting scheduler" tickInterval=10s maxAttempts=1 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ticker t=2024-09-10T04:01:30.857340607Z level=info msg=starting first_tick=2024-09-10T04:01:40Z 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=http.server t=2024-09-10T04:01:30.859270895Z level=info msg="HTTP Server TLS settings" MinTLSVersion=TLS1.2 configuredciphers=TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=http.server t=2024-09-10T04:01:30.859520606Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T04:01:30.928 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.dashboard t=2024-09-10T04:01:30.891249011Z level=info msg="starting to provision dashboards" 2024-09-10T04:01:30.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore.transactions t=2024-09-10T04:01:30.896005648Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T04:01:30.929 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore.transactions t=2024-09-10T04:01:30.915081867Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T04:01:31.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: Deploying daemon node-exporter.a on smithi038 2024-09-10T04:01:31.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: Deploying daemon node-exporter.a on smithi038 2024-09-10T04:01:31.007 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:30 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:31.008 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:30 smithi038 systemd[1]: Starting Ceph node-exporter.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:31.184 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=plugins.update.checker t=2024-09-10T04:01:30.93660204Z level=info msg="Update check succeeded" duration=79.913829ms 2024-09-10T04:01:31.184 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=grafana.update.checker t=2024-09-10T04:01:30.937920694Z level=info msg="Update check succeeded" duration=80.472427ms 2024-09-10T04:01:31.185 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=provisioning.dashboard t=2024-09-10T04:01:31.067488406Z level=info msg="finished to provision dashboards" 2024-09-10T04:01:31.185 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:31.449 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:31 smithi038 bash[56945]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-10T04:01:31.555 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=grafana-apiserver t=2024-09-10T04:01:31.184384449Z level=info msg="Adding GroupVersion playlist.grafana.app v0alpha1 to ResourceManager" 2024-09-10T04:01:31.555 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:01:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=grafana-apiserver t=2024-09-10T04:01:31.184983495Z level=info msg="Adding GroupVersion featuretoggle.grafana.app v0alpha1 to ResourceManager" 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:agent 2/2 51s ago 36s * 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:alertmanager ?:9093,9094 0/1 - 59s smithi038=a;count:1 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:grafana ?:3000 0/1 - 55s smithi050=a;count:1 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:iscsi.datapool ?:5000 1/1 51s ago 65s smithi050=iscsi.a;count:1 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 2/2 51s ago 5m smithi038=y;smithi050=x;count:2 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:mon 3/3 51s ago 5m smithi038:172.21.15.38=a;smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c;smithi050:172.21.15.50=b;count:3 2024-09-10T04:01:31.857 INFO:teuthology.orchestra.run.smithi038.stdout:node-exporter ?:9100 0/2 - 61s smithi038=a;smithi050=b;count:2 2024-09-10T04:01:31.858 INFO:teuthology.orchestra.run.smithi038.stdout:osd 8 51s ago - 2024-09-10T04:01:31.858 INFO:teuthology.orchestra.run.smithi038.stdout:prometheus ?:9095 0/1 - 63s smithi050=a;count:1 2024-09-10T04:01:31.858 INFO:teuthology.orchestra.run.smithi038.stdout:rgw.foo.a ?:80 1/1 44s ago 70s smithi038=foo.a;count:1 2024-09-10T04:01:31.858 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:31 smithi038 ceph-mon[25393]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:31.858 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:31 smithi038 ceph-mon[25393]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:31.858 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:31 smithi038 ceph-mon[30767]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:31.858 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:31 smithi038 ceph-mon[30767]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:31.859 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:31 smithi038 bash[56945]: Getting image source signatures 2024-09-10T04:01:31.859 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:31 smithi038 bash[56945]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-10T04:01:31.859 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:31 smithi038 bash[56945]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-10T04:01:31.859 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:31 smithi038 bash[56945]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-10T04:01:31.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:31 smithi050 ceph-mon[26687]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:31.946 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:31 smithi050 ceph-mon[26687]: pgmap v21: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:32.787 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T04:01:33.383 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:33.707 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:33 smithi038 bash[56945]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-10T04:01:33.708 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:33 smithi038 bash[56945]: Writing manifest to image destination 2024-09-10T04:01:33.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:33 smithi038 ceph-mon[25393]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:33.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:33 smithi038 ceph-mon[25393]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:33.962 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:33 smithi038 ceph-mon[30767]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:33.962 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:33 smithi038 ceph-mon[30767]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:34.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:33 smithi050 ceph-mon[26687]: from='client.24653 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:34.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:33 smithi050 ceph-mon[26687]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:34.249 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-09-10T04:01:34.250 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-09-10T04:01:34.250 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 172.21.15.50 2024-09-10T04:01:34.250 INFO:teuthology.orchestra.run.smithi038.stdout:2 hosts in cluster 2024-09-10T04:01:34.250 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:34 smithi038 podman[56945]: 2024-09-10 04:01:34.156872638 +0000 UTC m=+3.096936659 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-10T04:01:34.250 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:34 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ::ffff:172.21.15.50 - - [10/Sep/2024:04:01:34] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-10T04:01:34.504 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:34 smithi038 podman[56945]: 2024-09-10 04:01:34.263418052 +0000 UTC m=+3.203482072 container create db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:34.975 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T04:01:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 podman[56945]: 2024-09-10 04:01:35.195667289 +0000 UTC m=+4.135731310 container init db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 podman[56945]: 2024-09-10 04:01:35.199329364 +0000 UTC m=+4.139393385 container start db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.202Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.202Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.203Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T04:01:35.479 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.203Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.203Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.203Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T04:01:35.480 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T04:01:35.481 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T04:01:35.482 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a[57242]: ts=2024-09-10T04:01:35.204Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 bash[56945]: db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 2024-09-10T04:01:35.483 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:01:35 smithi038 systemd[1]: Started Ceph node-exporter.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:01:35.801 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[25393]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[25393]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[25393]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[30767]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:36.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[30767]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:36.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.006 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:35 smithi038 ceph-mon[30767]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 68s ago Has a FileSystem, LVM detected 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme1n1 ssd Linux_ae77412bb688adfead48 89.4G No 68s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme2n1 ssd Linux_6ae6bc22a0fc64d8701b 89.4G No 68s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme3n1 ssd Linux_15503f945018a7b16b2d 89.4G No 68s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme4n1 ssd Linux_f3a6674ebb3c7a1c1136 89.4G No 68s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W4J33D 931G No 68s ago Has partitions 2024-09-10T04:01:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513001F480DGN 447G No 78s ago Has a FileSystem, LVM detected 2024-09-10T04:01:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/nvme1n1 ssd Linux_ea81da236ed925dfb7ac 89.4G No 78s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/nvme2n1 ssd Linux_fb829e2adb144ce37343 89.4G No 78s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/nvme3n1 ssd Linux_5996ec7cb7dc2a61b541 89.4G No 78s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/nvme4n1 ssd Linux_e2d852fcc8ac72dd9d09 89.4G No 78s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T04:01:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:smithi050 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W4HNFD 931G No 78s ago Has partitions 2024-09-10T04:01:36.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-mon[26687]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:36.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-mon[26687]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:36.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:35 smithi050 ceph-mon[26687]: from='mgr.24485 ' entity='mgr.y' 2024-09-10T04:01:36.735 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-10T04:01:37.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:36 smithi038 ceph-mon[25393]: Deploying daemon node-exporter.b on smithi050 2024-09-10T04:01:37.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:36 smithi038 ceph-mon[25393]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:37.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:36 smithi038 ceph-mon[30767]: Deploying daemon node-exporter.b on smithi050 2024-09-10T04:01:37.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:36 smithi038 ceph-mon[30767]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:37.010 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:37.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:36 smithi050 ceph-mon[26687]: Deploying daemon node-exporter.b on smithi050 2024-09-10T04:01:37.055 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:36 smithi050 ceph-mon[26687]: from='client.14571 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T04:01:37.740 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: agent 2024-09-10T04:01:37.740 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: agent 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: host_pattern: '*' 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:54.902298Z' 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:40.223321Z' 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: running: 2 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: size: 2 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:events: 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:- 2024-09-10T04:00:56.118733Z service:agent [INFO] "service was created" 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: alertmanager 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: alertmanager 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.741 INFO:teuthology.orchestra.run.smithi038.stdout: count: 1 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038=a 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:32.576913Z' 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: - 9093 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: - 9094 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: running: 0 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout: size: 1 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout:events: 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout:- 2024-09-10T04:01:17.622645Z service:alertmanager [INFO] "service was created" 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.742 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: grafana 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: grafana 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: count: 1 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050=a 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout:spec: 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: anonymous_access: true 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: protocol: https 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:36.528333Z' 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: - 3000 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: running: 0 2024-09-10T04:01:37.743 INFO:teuthology.orchestra.run.smithi038.stdout: size: 1 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:events: 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:- 2024-09-10T04:01:29.814849Z service:grafana [INFO] "service was created" 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: iscsi 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:service_id: datapool 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: iscsi.datapool 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: count: 1 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050=iscsi.a 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout:spec: 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: api_password: admin 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: api_port: 5000 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: api_user: admin 2024-09-10T04:01:37.744 INFO:teuthology.orchestra.run.smithi038.stdout: pool: datapool 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: trusted_ip_list: 172.21.15.50 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:26.724940Z' 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:40.224007Z' 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: - 5000 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: running: 1 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: size: 1 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: mgr 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: mgr 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: count: 2 2024-09-10T04:01:37.745 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038=y 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050=x 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T03:56:20.161700Z' 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:40.223542Z' 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: running: 2 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: size: 2 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: mon 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: mon 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: count: 3 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.746 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038:172.21.15.38=a 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038:[v2:172.21.15.38:3301,v1:172.21.15.38:6790]=c 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050:172.21.15.50=b 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T03:55:55.573109Z' 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:40.223443Z' 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: running: 3 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: size: 3 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: node-exporter 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: node-exporter 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: count: 2 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.747 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038=a 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050=b 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:30.101092Z' 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: - 9100 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: running: 0 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout: size: 2 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:events: 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:- 2024-09-10T04:01:35.346793Z service:node-exporter [INFO] "service was created" 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: osd 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: osd 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:unmanaged: true 2024-09-10T04:01:37.748 INFO:teuthology.orchestra.run.smithi038.stdout:spec: 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: filter_logic: AND 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: objectstore: bluestore 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: container_image_id: d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:40.223633Z' 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: running: 8 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: size: 8 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: prometheus 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: prometheus 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: count: 1 2024-09-10T04:01:37.749 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi050=a 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:28.417174Z' 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: - 9095 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: running: 0 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: size: 1 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:--- 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:service_type: rgw 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:service_id: foo.a 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:service_name: rgw.foo.a 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout:placement: 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: count: 1 2024-09-10T04:01:37.750 INFO:teuthology.orchestra.run.smithi038.stdout: hosts: 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: - smithi038=foo.a 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout:status: 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: created: '2024-09-10T04:00:21.492775Z' 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: last_refresh: '2024-09-10T04:00:47.216885Z' 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: ports: 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: - 80 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: running: 1 2024-09-10T04:01:37.751 INFO:teuthology.orchestra.run.smithi038.stdout: size: 1 2024-09-10T04:01:38.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:37 smithi038 ceph-mon[25393]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:38.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:37 smithi038 ceph-mon[25393]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:38.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:37 smithi038 ceph-mon[30767]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:38.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:37 smithi038 ceph-mon[30767]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:38.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:37 smithi050 ceph-mon[26687]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T04:01:38.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:37 smithi050 ceph-mon[26687]: pgmap v24: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:39.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:38 smithi038 ceph-mon[25393]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T04:01:39.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:38 smithi038 ceph-mon[30767]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T04:01:39.172 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-10T04:01:39.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:38 smithi050 ceph-mon[26687]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T04:01:40.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:39 smithi038 ceph-mon[25393]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:40.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:39 smithi038 ceph-mon[30767]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:40.304 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:39 smithi050 ceph-mon[26687]: pgmap v25: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T04:01:40.377 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T04:01:41.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:40 smithi038 ceph-mon[25393]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:41.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:40 smithi038 ceph-mon[30767]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:41.063 INFO:teuthology.orchestra.run.smithi038.stdout:osd 8 60s ago - 2024-09-10T04:01:41.305 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:40 smithi050 ceph-mon[26687]: from='mgr.24485 172.21.15.38:0/669345550' entity='mgr.y' cmd={"prefix": "osd blocklist ls", "format": "json"} : dispatch 2024-09-10T04:01:41.305 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:40 smithi050 bash[49579]: Trying to pull quay.io/prometheus/node-exporter:v1.7.0... 2024-09-10T04:01:41.682 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:41 smithi050 bash[49579]: Getting image source signatures 2024-09-10T04:01:41.682 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:41 smithi050 bash[49579]: Copying blob sha256:324153f2810a9927fcce320af9e4e291e0b6e805cbdd1f338386c756b9defa24 2024-09-10T04:01:41.682 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:41 smithi050 bash[49579]: Copying blob sha256:2abcce694348cd2c949c0e98a7400ebdfd8341021bcf6b541bc72033ce982510 2024-09-10T04:01:41.682 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:41 smithi050 bash[49579]: Copying blob sha256:455fd88e5221bc1e278ef2d059cd70e4df99a24e5af050ede621534276f6cf9a 2024-09-10T04:01:41.743 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T04:01:41.754 INFO:tasks.cephadm:Teardown begin 2024-09-10T04:01:41.754 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:01:41.786 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:01:41.816 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T04:01:41.816 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T04:01:41.843 DEBUG:teuthology.orchestra.run.smithi050:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T04:01:41.873 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T04:01:41.873 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T04:01:41.874 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a 2024-09-10T04:01:42.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:41 smithi038 ceph-mon[30767]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:42.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:41 smithi038 ceph-mon[25393]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:42.005 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:41 smithi038 systemd[1]: Stopping Ceph mon.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:42.054 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:41 smithi050 ceph-mon[26687]: pgmap v26: 132 pgs: 132 active+clean; 583 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T04:01:42.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:42 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a[25389]: 2024-09-10T04:01:42.365+0000 7f2e65db0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:42.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:42 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a[25389]: 2024-09-10T04:01:42.365+0000 7f2e65db0640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T04:01:42.754 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:42 smithi038 podman[57811]: 2024-09-10 04:01:42.457555179 +0000 UTC m=+0.204422045 container died c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T04:01:43.054 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:42 smithi050 bash[49579]: Copying config sha256:72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e 2024-09-10T04:01:43.055 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:42 smithi050 bash[49579]: Writing manifest to image destination 2024-09-10T04:01:43.166 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:42 smithi038 podman[57811]: 2024-09-10 04:01:42.864979465 +0000 UTC m=+0.611846330 container cleanup c626664d8371b8b997ea3238da4ce24d0e003c8ebbae698c8224d3ed61c0d4aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T04:01:43.167 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 10 04:01:42 smithi038 bash[57811]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-a 2024-09-10T04:01:43.311 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T04:01:43.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:43.365 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T04:01:43.365 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T04:01:43.366 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.c 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 podman[49579]: 2024-09-10 04:01:43.033929824 +0000 UTC m=+2.141716677 image pull 72c9c208898624938c9e4183d6686ea4a5fd3f912bc29bc3f00147924c521a3e quay.io/prometheus/node-exporter:v1.7.0 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 podman[49579]: 2024-09-10 04:01:43.148854566 +0000 UTC m=+2.256641407 container create 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 podman[49579]: 2024-09-10 04:01:43.334355 +0000 UTC m=+2.442141840 container init 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 podman[49579]: 2024-09-10 04:01:43.337465621 +0000 UTC m=+2.445252473 container start 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.341Z caller=node_exporter.go:192 level=info msg="Starting node_exporter" version="(version=1.7.0, branch=HEAD, revision=7333465abf9efba81876303bb57e6fadb946041b)" 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.341Z caller=node_exporter.go:193 level=info msg="Build context" build_context="(go=go1.21.4, platform=linux/amd64, user=root@35918982f6d8, date=20231112-23:53:35, tags=netgo osusergo static_build)" 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T04:01:43.402 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=diskstats_linux.go:265 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T04:01:43.403 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T04:01:43.404 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.342Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T04:01:43.405 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=tls_config.go:274 level=info msg="Listening on" address=[::]:9100 2024-09-10T04:01:43.406 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b[49698]: ts=2024-09-10T04:01:43.343Z caller=tls_config.go:277 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T04:01:43.407 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 bash[49579]: 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e 2024-09-10T04:01:43.504 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:43 smithi038 systemd[1]: Stopping Ceph mon.c for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:43.805 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:01:43 smithi050 systemd[1]: Started Ceph node-exporter.b for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:01:44.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c[30763]: 2024-09-10T04:01:43.661+0000 7fcda44ef640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:44.004 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c[30763]: 2024-09-10T04:01:43.661+0000 7fcda44ef640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:43 smithi038 podman[58048]: 2024-09-10 04:01:43.784051257 +0000 UTC m=+0.217369921 container died 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE Bus STOPPING 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE Bus STOPPED 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE Bus STARTING 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE Serving on http://:::9283 2024-09-10T04:01:44.005 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:43 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: [10/Sep/2024:04:01:43] ENGINE Bus STARTED 2024-09-10T04:01:44.504 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:44 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: ::ffff:172.21.15.50 - - [10/Sep/2024:04:01:44] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.51.0" 2024-09-10T04:01:44.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:44 smithi038 podman[58048]: 2024-09-10 04:01:44.228534821 +0000 UTC m=+0.661853470 container cleanup 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T04:01:44.505 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:44 smithi038 bash[58048]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c 2024-09-10T04:01:44.798 INFO:journalctl@ceph.mon.c.smithi038.stdout:Sep 10 04:01:44 smithi038 podman[58085]: 2024-09-10 04:01:44.513034907 +0000 UTC m=+0.733605069 container remove 891472313ee98e3213dfc6c335376a2293d7f50d0281f29385b401c3df8506d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T04:01:45.303 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T04:01:45.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:45.342 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T04:01:45.342 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T04:01:45.342 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.b 2024-09-10T04:01:45.705 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:01:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug there is no tcmu-runner data available 2024-09-10T04:01:45.705 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:45 smithi050 systemd[1]: Stopping Ceph mon.b for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:46.038 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b[26679]: 2024-09-10T04:01:45.703+0000 7f4ae3aec640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:46.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b[26679]: 2024-09-10T04:01:45.703+0000 7f4ae3aec640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T04:01:46.039 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:45 smithi050 podman[49757]: 2024-09-10 04:01:45.78629443 +0000 UTC m=+0.243580224 container died 432ee69f77706bd38cc4c193de13cc4e78df537918e5b881cd78fa4dfd53004d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD) 2024-09-10T04:01:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:46 smithi050 podman[49757]: 2024-09-10 04:01:46.415175812 +0000 UTC m=+0.872461607 container cleanup 432ee69f77706bd38cc4c193de13cc4e78df537918e5b881cd78fa4dfd53004d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2) 2024-09-10T04:01:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:46 smithi050 bash[49757]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b 2024-09-10T04:01:46.805 INFO:journalctl@ceph.mon.b.smithi050.stdout:Sep 10 04:01:46 smithi050 podman[49769]: 2024-09-10 04:01:46.608443924 +0000 UTC m=+0.820087825 container remove 432ee69f77706bd38cc4c193de13cc4e78df537918e5b881cd78fa4dfd53004d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mon-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:01:46.855 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T04:01:46.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:46.905 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T04:01:46.905 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T04:01:46.905 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y 2024-09-10T04:01:47.254 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:46 smithi038 systemd[1]: Stopping Ceph mgr.y for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:47.754 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:47 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y[25623]: 2024-09-10T04:01:47.467+0000 7fa7eed15640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:47.754 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:47 smithi038 podman[58560]: 2024-09-10 04:01:47.499639971 +0000 UTC m=+0.146944533 container died 6741900d5e6b9880ae7a63c49301923f5518959ee0b8189fea9c68193ce7046b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T04:01:47.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:01:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:01:47.633Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:01:47.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:01:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:01:47.635Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:01:47.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:01:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:01:47.635Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:01:47.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:01:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:01:47.636Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:01:47.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:01:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:01:47.636Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:01:48.254 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:47 smithi038 podman[58560]: 2024-09-10 04:01:47.976811365 +0000 UTC m=+0.624115967 container cleanup 6741900d5e6b9880ae7a63c49301923f5518959ee0b8189fea9c68193ce7046b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3a5e3f16720f432aea965c40a8f6eb480095ec6c, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T04:01:48.254 INFO:journalctl@ceph.mgr.y.smithi038.stdout:Sep 10 04:01:47 smithi038 bash[58560]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-y 2024-09-10T04:01:48.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T04:01:48.602 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:48.602 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T04:01:48.603 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T04:01:48.603 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.x 2024-09-10T04:01:48.952 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:01:48 smithi050 systemd[1]: Stopping Ceph mgr.x for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:49.247 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:01:48 smithi050 podman[49880]: 2024-09-10 04:01:48.951778096 +0000 UTC m=+0.149515881 container died f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T04:01:49.554 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:01:49 smithi050 podman[49880]: 2024-09-10 04:01:49.428982097 +0000 UTC m=+0.626719873 container cleanup f5e92250f36e35514c521bcc51f7cd852304537b69154d8eb208a3e405616534 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T04:01:49.555 INFO:journalctl@ceph.mgr.x.smithi050.stdout:Sep 10 04:01:49 smithi050 bash[49880]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-mgr-x 2024-09-10T04:01:49.798 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T04:01:49.834 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:49.834 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T04:01:49.834 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T04:01:49.835 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.0 2024-09-10T04:01:50.166 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:49 smithi038 systemd[1]: Stopping Ceph osd.0 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:50.504 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:50 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0[34601]: 2024-09-10T04:01:50.164+0000 7f70cb5e2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:50.504 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:50 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0[34601]: 2024-09-10T04:01:50.164+0000 7f70cb5e2640 -1 osd.0 71 *** Got signal Terminated *** 2024-09-10T04:01:50.504 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:50 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0[34601]: 2024-09-10T04:01:50.164+0000 7f70cb5e2640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:01:56.005 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:55 smithi038 podman[58697]: 2024-09-10 04:01:55.733699781 +0000 UTC m=+5.706449154 container died 641be81577e25c28a4e155bf9ce634e7576e71f05f5b531574caf454dcce4980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T04:01:56.523 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:56 smithi038 podman[58697]: 2024-09-10 04:01:56.266872649 +0000 UTC m=+6.239622004 container cleanup 641be81577e25c28a4e155bf9ce634e7576e71f05f5b531574caf454dcce4980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0) 2024-09-10T04:01:56.524 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:56 smithi038 bash[58697]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0 2024-09-10T04:01:56.524 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:56 smithi038 podman[58773]: 2024-09-10 04:01:56.502116803 +0000 UTC m=+0.819876486 container remove 641be81577e25c28a4e155bf9ce634e7576e71f05f5b531574caf454dcce4980 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:01:57.423 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.303775697 +0000 UTC m=+0.016388316 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:01:57.700 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.422615373 +0000 UTC m=+0.135228031 container create 4243ce6f5c8087c535e8a2d76e681b91c010818af1048220385cc5367399301b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:01:57.700 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.616500582 +0000 UTC m=+0.329113192 container init 4243ce6f5c8087c535e8a2d76e681b91c010818af1048220385cc5367399301b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T04:01:57.700 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.619905956 +0000 UTC m=+0.332518572 container start 4243ce6f5c8087c535e8a2d76e681b91c010818af1048220385cc5367399301b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T04:01:58.004 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.699773144 +0000 UTC m=+0.412385756 container attach 4243ce6f5c8087c535e8a2d76e681b91c010818af1048220385cc5367399301b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:01:58.004 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 10 04:01:57 smithi038 podman[58938]: 2024-09-10 04:01:57.823824441 +0000 UTC m=+0.536437051 container died 4243ce6f5c8087c535e8a2d76e681b91c010818af1048220385cc5367399301b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0) 2024-09-10T04:01:58.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T04:01:58.538 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:01:58.538 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T04:01:58.538 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T04:01:58.538 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.1 2024-09-10T04:01:58.754 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:01:58 smithi038 systemd[1]: Stopping Ceph osd.1 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:01:59.254 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:01:58 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1[37994]: 2024-09-10T04:01:58.880+0000 7f888a212640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:01:59.254 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:01:58 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1[37994]: 2024-09-10T04:01:58.880+0000 7f888a212640 -1 osd.1 71 *** Got signal Terminated *** 2024-09-10T04:01:59.254 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:01:58 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1[37994]: 2024-09-10T04:01:58.880+0000 7f888a212640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:04.504 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:04 smithi038 podman[59039]: 2024-09-10 04:02:04.374545881 +0000 UTC m=+5.599966763 container died a70f4bfcec71a316e3f1d9ce8df6e8a80ebaf42fbfea3124dce669f203d1430c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:05.004 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:04 smithi038 podman[59039]: 2024-09-10 04:02:04.813190101 +0000 UTC m=+6.038610973 container cleanup a70f4bfcec71a316e3f1d9ce8df6e8a80ebaf42fbfea3124dce669f203d1430c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T04:02:05.004 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:04 smithi038 bash[59039]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1 2024-09-10T04:02:05.754 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:05 smithi038 podman[59522]: 2024-09-10 04:02:05.441584054 +0000 UTC m=+1.185167623 container remove a70f4bfcec71a316e3f1d9ce8df6e8a80ebaf42fbfea3124dce669f203d1430c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T04:02:06.164 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:05 smithi038 bash[59630]: Error: no container with name or ID "ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1" found: no such container 2024-09-10T04:02:06.164 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:06.035942213 +0000 UTC m=+0.100605783 container create 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:06.164 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:05.951823995 +0000 UTC m=+0.016487533 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:06.476 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:06.271497095 +0000 UTC m=+0.336160631 container init 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T04:02:06.476 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:06.275171467 +0000 UTC m=+0.339834995 container start 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:02:06.476 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:06.354725109 +0000 UTC m=+0.419388647 container attach 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T04:02:06.476 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:06 smithi038 podman[59712]: 2024-09-10 04:02:06.475342104 +0000 UTC m=+0.540005647 container died 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-10T04:02:07.023 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T04:02:07.047 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:07 smithi038 podman[59736]: 2024-09-10 04:02:07.000023134 +0000 UTC m=+0.518821696 container remove 8412054706b5475396daed0dc12d0bcf7bf4e8841dd055aceae753122ca62f90 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:02:07.048 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:07 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-10T04:02:07.048 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:07 smithi038 systemd[1]: Stopped Ceph osd.1 for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:02:07.048 INFO:journalctl@ceph.osd.1.smithi038.stdout:Sep 10 04:02:07 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.1.service: Consumed 3.341s CPU time. 2024-09-10T04:02:07.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:07.064 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T04:02:07.064 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T04:02:07.064 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.2 2024-09-10T04:02:07.392 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:07 smithi038 systemd[1]: Stopping Ceph osd.2 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:07.754 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2[41372]: 2024-09-10T04:02:07.396+0000 7fc057c86640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:07.754 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2[41372]: 2024-09-10T04:02:07.396+0000 7fc057c86640 -1 osd.2 71 *** Got signal Terminated *** 2024-09-10T04:02:07.754 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:07 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2[41372]: 2024-09-10T04:02:07.396+0000 7fc057c86640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:13.005 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:12 smithi038 podman[59827]: 2024-09-10 04:02:12.745088798 +0000 UTC m=+5.454489578 container died 3e4de0aa949787e9f04eb1b91b31b4bc55b76f129a3bc5af5b76e68c4b16c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T04:02:13.294 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:13 smithi038 podman[59827]: 2024-09-10 04:02:13.173850005 +0000 UTC m=+5.883250782 container cleanup 3e4de0aa949787e9f04eb1b91b31b4bc55b76f129a3bc5af5b76e68c4b16c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T04:02:13.294 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:13 smithi038 bash[59827]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2 2024-09-10T04:02:13.609 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:13 smithi038 podman[59842]: 2024-09-10 04:02:13.393040784 +0000 UTC m=+0.642495456 container remove 3e4de0aa949787e9f04eb1b91b31b4bc55b76f129a3bc5af5b76e68c4b16c726 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:14.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:02:13 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=infra.usagestats t=2024-09-10T04:02:13.863539934Z level=info msg="Usage stats are ready to report" 2024-09-10T04:02:15.287 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:14.910524892 +0000 UTC m=+0.016819299 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:15.288 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:15.023975001 +0000 UTC m=+0.130269413 container create 464c76201d246e661cfe1ebc89cc4ed648a4edf76165d111a5945edad1c75954 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T04:02:15.288 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:15.286479489 +0000 UTC m=+0.392773902 container init 464c76201d246e661cfe1ebc89cc4ed648a4edf76165d111a5945edad1c75954 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:15.679 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:15.290834383 +0000 UTC m=+0.397128799 container start 464c76201d246e661cfe1ebc89cc4ed648a4edf76165d111a5945edad1c75954 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T04:02:15.679 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:15.394829969 +0000 UTC m=+0.501124388 container attach 464c76201d246e661cfe1ebc89cc4ed648a4edf76165d111a5945edad1c75954 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:15.679 INFO:journalctl@ceph.osd.2.smithi038.stdout:Sep 10 04:02:15 smithi038 podman[59994]: 2024-09-10 04:02:15.498198772 +0000 UTC m=+0.604493187 container died 464c76201d246e661cfe1ebc89cc4ed648a4edf76165d111a5945edad1c75954 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T04:02:16.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:15 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:15.996+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:16.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T04:02:16.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:16.425 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T04:02:16.425 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T04:02:16.425 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.3 2024-09-10T04:02:16.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:16 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:16.400+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:16.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:16 smithi038 systemd[1]: Stopping Ceph osd.3 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:16.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:16 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:16.352+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:17.304 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:17 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:17.004+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:17.436 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:17.043+0000 7fe75757a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:17.436 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:17.043+0000 7fe75757a640 -1 osd.3 71 *** Got signal Terminated *** 2024-09-10T04:02:17.436 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:17.043+0000 7fe75757a640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:17.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:17 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:17.434+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:17.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:17 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:17.371+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:17.805 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:17 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:17.348+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:18.304 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:17 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:17.969+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:18.691 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:18 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:18.308+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:18.691 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:18 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:18.401+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:18.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:18 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:18.431+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:19.304 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:19 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:19.008+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:19.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:19 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:19.424+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:19.804 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:19 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:19.560+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:19.805 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:19 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:19.358+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:19.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:19 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:19.403+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:20.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:20 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:20.012+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:20.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:20 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:20.418+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:20.775 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:20 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:20.556+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:20.775 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:20 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:20.339+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:20.776 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:20 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:20.418+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:21.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:20 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:20.982+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:21.555 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:21 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:21.518+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:21.555 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:21 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:21.406+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:21.555 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:21 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:21.296+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:21.754 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:21 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3[44903]: 2024-09-10T04:02:21.409+0000 7fe74fb87640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:51.594256+0000 front 2024-09-10T04:01:51.594639+0000 (oldest deadline 2024-09-10T04:02:16.294319+0000) 2024-09-10T04:02:22.304 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:22 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:22.253+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:22.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:22 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:22.009+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:22.689 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:22 smithi038 podman[60172]: 2024-09-10 04:02:22.429845373 +0000 UTC m=+5.477385169 container died bddd0e12876cd35e4c1b3237cc48b2e8f86d4b017528e0bab5aa170eb6c75897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:02:22.804 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:22 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:22.513+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:22.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:22 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:22.385+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:23.004 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:22 smithi038 podman[60172]: 2024-09-10 04:02:22.846801865 +0000 UTC m=+5.894341665 container cleanup bddd0e12876cd35e4c1b3237cc48b2e8f86d4b017528e0bab5aa170eb6c75897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:23.004 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:22 smithi038 bash[60172]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3 2024-09-10T04:02:23.308 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:23 smithi038 podman[60611]: 2024-09-10 04:02:23.006289382 +0000 UTC m=+0.626850936 container remove bddd0e12876cd35e4c1b3237cc48b2e8f86d4b017528e0bab5aa170eb6c75897 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:02:23.347 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:23 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:23.252+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:23.347 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:23 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:23.346+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:23.347 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:23 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:23.058+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:23.804 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:23 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:23.551+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:24.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:24 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:24.222+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:24.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:24 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:24.026+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:24.728 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:24 smithi038 podman[60861]: 2024-09-10 04:02:24.49842512 +0000 UTC m=+0.016631051 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:24.728 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:24 smithi038 podman[60861]: 2024-09-10 04:02:24.597790586 +0000 UTC m=+0.115996507 container create e820a941525981f4c07b97aefc8e63a5ee8e37608294e42243f5477f248b915b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:02:24.805 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:24 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:24.578+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:24.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:24 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:24.373+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:25.093 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:24 smithi038 podman[60861]: 2024-09-10 04:02:24.833283756 +0000 UTC m=+0.351489682 container init e820a941525981f4c07b97aefc8e63a5ee8e37608294e42243f5477f248b915b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-10T04:02:25.093 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:24 smithi038 podman[60861]: 2024-09-10 04:02:24.836644455 +0000 UTC m=+0.354850377 container start e820a941525981f4c07b97aefc8e63a5ee8e37608294e42243f5477f248b915b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T04:02:25.093 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:24 smithi038 podman[60861]: 2024-09-10 04:02:24.966646399 +0000 UTC m=+0.484852329 container attach e820a941525981f4c07b97aefc8e63a5ee8e37608294e42243f5477f248b915b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T04:02:25.093 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 10 04:02:25 smithi038 podman[60861]: 2024-09-10 04:02:25.057721077 +0000 UTC m=+0.575927009 container died e820a941525981f4c07b97aefc8e63a5ee8e37608294e42243f5477f248b915b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T04:02:25.304 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:25 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:25.207+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:25.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:25 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:25.033+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:25.727 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T04:02:25.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:25.770 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T04:02:25.771 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T04:02:25.771 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.4 2024-09-10T04:02:25.802 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:25 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:25.529+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:25.802 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:25 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:25.363+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:26.054 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:25 smithi050 systemd[1]: Stopping Ceph osd.4 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:26.360 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:26.073+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:26.361 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:26.207+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:26.361 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:26.122+0000 7f5aa01d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:26.361 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:26.122+0000 7f5aa01d4640 -1 osd.4 71 *** Got signal Terminated *** 2024-09-10T04:02:26.361 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:26.122+0000 7f5aa01d4640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:26.804 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:26.359+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:26.805 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:26.531+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:26.805 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:26 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:26.531+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:00.233254+0000 front 2024-09-10T04:02:00.233277+0000 (oldest deadline 2024-09-10T04:02:26.133028+0000) 2024-09-10T04:02:27.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:27.033+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:27.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:27.218+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:27.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:27.218+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:27.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:27.382+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:27.806 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:27.549+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:27.806 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:27 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:27.549+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:00.233254+0000 front 2024-09-10T04:02:00.233277+0000 (oldest deadline 2024-09-10T04:02:26.133028+0000) 2024-09-10T04:02:28.304 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:28.028+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:28.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:28.170+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:28.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:28.170+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:28.710 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:28.581+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:28.710 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:28.581+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:00.233254+0000 front 2024-09-10T04:02:00.233277+0000 (oldest deadline 2024-09-10T04:02:26.133028+0000) 2024-09-10T04:02:28.711 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:28 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:28.399+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:29.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:29.065+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:29.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:29.210+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:29.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:29.210+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:29.775 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:29.592+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:29.775 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:29.592+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:00.233254+0000 front 2024-09-10T04:02:00.233277+0000 (oldest deadline 2024-09-10T04:02:26.133028+0000) 2024-09-10T04:02:29.776 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:29.410+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:29.776 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:29 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:29.410+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:30.304 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:30.187+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:30.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:30.187+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:30.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:30.027+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:30.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:30.027+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:30.629 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:30.438+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:30.630 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:30.438+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:30.907 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:30.628+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:50.932021+0000 front 2024-09-10T04:01:50.931974+0000 (oldest deadline 2024-09-10T04:02:19.132367+0000) 2024-09-10T04:02:30.907 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:30 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4[32506]: 2024-09-10T04:02:30.628+0000 7f5a9c7e9640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:00.233254+0000 front 2024-09-10T04:02:00.233277+0000 (oldest deadline 2024-09-10T04:02:26.133028+0000) 2024-09-10T04:02:31.269 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:31.147+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:31.269 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:31.147+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:31.269 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:31.003+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:31.270 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:31.003+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:31.552 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:31.434+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:31.552 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:31 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:31.434+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:31.804 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:31 smithi050 podman[51019]: 2024-09-10 04:02:31.587595958 +0000 UTC m=+5.612174470 container died ddd2d2153b252d319d77c2dd5c891a20c61085d2bfefc357a099f0159376e87d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T04:02:32.145 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:31 smithi050 podman[51019]: 2024-09-10 04:02:31.930107517 +0000 UTC m=+5.954686016 container cleanup ddd2d2153b252d319d77c2dd5c891a20c61085d2bfefc357a099f0159376e87d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T04:02:32.145 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:31 smithi050 bash[51019]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4 2024-09-10T04:02:32.145 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:32.018+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:32.145 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:32.018+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:32.506 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51188]: 2024-09-10 04:02:32.181406485 +0000 UTC m=+0.605378432 container remove ddd2d2153b252d319d77c2dd5c891a20c61085d2bfefc357a099f0159376e87d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T04:02:32.507 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:32.144+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:32.507 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:32.144+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:32.507 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:32.391+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:32.507 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:32.391+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:32.785 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.408766955 +0000 UTC m=+0.016316539 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:32.786 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.507358128 +0000 UTC m=+0.114907697 container create 3c93c005b936962f90ca477b3a487a467738817d96c2a77ec9be01f495b1b9d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-10T04:02:32.786 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.6932699 +0000 UTC m=+0.300819470 container init 3c93c005b936962f90ca477b3a487a467738817d96c2a77ec9be01f495b1b9d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T04:02:32.786 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.696683036 +0000 UTC m=+0.304232606 container start 3c93c005b936962f90ca477b3a487a467738817d96c2a77ec9be01f495b1b9d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:32.786 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.785208762 +0000 UTC m=+0.392758336 container attach 3c93c005b936962f90ca477b3a487a467738817d96c2a77ec9be01f495b1b9d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T04:02:33.055 INFO:journalctl@ceph.osd.4.smithi050.stdout:Sep 10 04:02:32 smithi050 podman[51281]: 2024-09-10 04:02:32.89425946 +0000 UTC m=+0.501809039 container died 3c93c005b936962f90ca477b3a487a467738817d96c2a77ec9be01f495b1b9d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T04:02:33.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:32.969+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:33.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:32.969+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:33.464 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:33.182+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:33.464 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:33.182+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:33.465 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.383+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:33.465 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.383+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:33.465 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.383+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:33.485 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T04:02:33.571 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:33.571 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T04:02:33.571 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T04:02:33.571 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.5 2024-09-10T04:02:33.804 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 systemd[1]: Stopping Ceph osd.5 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:34.153 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:33.996+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:34.153 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:33.996+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:34.153 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:33.996+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:34.154 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.892+0000 7f808a854640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:34.154 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.892+0000 7f808a854640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-10T04:02:34.154 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:33 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:33.892+0000 7f808a854640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:34.555 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:34.375+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:34.555 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:34.375+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:34.555 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:34.375+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:34.556 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:34.151+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:34.556 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:34.151+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:34.556 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:34 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:34.151+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:35.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:35.121+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:35.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:35.122+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:35.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:35.122+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:35.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:35.026+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:35.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:35.026+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:35.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:35.026+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:35.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:35.342+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:35.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:35.342+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:35.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:35 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:35.342+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:36.019+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:36.019+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:36.019+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:36.098+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:36.098+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:36.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:36.098+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:36.804 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:36.324+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:36.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:36.324+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:36.805 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:36 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:36.324+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.031+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.031+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.031+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.031+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:37.109+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:37.109+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:37.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:37.109+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:37.632 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:37.374+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:37.632 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:37.374+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:37.632 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:37.374+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:38.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:38.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:38.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.989+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.990+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:38.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.990+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:38.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:37 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:37.990+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:38.564 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:38.384+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.475699+0000 front 2024-09-10T04:01:53.475633+0000 (oldest deadline 2024-09-10T04:02:16.275853+0000) 2024-09-10T04:02:38.564 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:38.384+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.877127+0000 front 2024-09-10T04:02:03.877250+0000 (oldest deadline 2024-09-10T04:02:28.577087+0000) 2024-09-10T04:02:38.565 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5[35758]: 2024-09-10T04:02:38.384+0000 7f8082e61640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:08.577771+0000 front 2024-09-10T04:02:08.577636+0000 (oldest deadline 2024-09-10T04:02:33.277558+0000) 2024-09-10T04:02:39.304 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:38.943+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:38.943+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:38.943+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:38 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:38.943+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:39.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:39.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:39.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:39.116+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:39.721 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:39 smithi050 podman[51383]: 2024-09-10 04:02:39.534866976 +0000 UTC m=+5.773675164 container died 16f82ad450c7791df0801de92e3002e27e13d33d09b5d7fe8693bd972691754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T04:02:40.029 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:39.904+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:40.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:39.904+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:40.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:39.904+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:40.030 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:39 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:39.904+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[51383]: 2024-09-10 04:02:40.028933652 +0000 UTC m=+6.267741838 container cleanup 16f82ad450c7791df0801de92e3002e27e13d33d09b5d7fe8693bd972691754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 bash[51383]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[52034]: 2024-09-10 04:02:40.222088106 +0000 UTC m=+0.816474883 container remove 16f82ad450c7791df0801de92e3002e27e13d33d09b5d7fe8693bd972691754d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:40.073+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:40.073+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:40.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:40.073+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:40.904 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:40.857+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:40.904 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:40.857+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:40.904 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:40.857+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:40.904 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:40 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:40.857+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:40.905 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[52146]: 2024-09-10 04:02:40.556147507 +0000 UTC m=+0.016525863 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:40.905 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[52146]: 2024-09-10 04:02:40.664670196 +0000 UTC m=+0.125048577 container create ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:02:40.905 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[52146]: 2024-09-10 04:02:40.900497836 +0000 UTC m=+0.360876179 container init ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T04:02:40.905 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:40 smithi050 podman[52146]: 2024-09-10 04:02:40.903937153 +0000 UTC m=+0.364315505 container start ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T04:02:41.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:41.087+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:41.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:41.087+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:41.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:41.087+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:41.305 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 podman[52146]: 2024-09-10 04:02:41.000443693 +0000 UTC m=+0.460822054 container attach ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T04:02:41.305 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 podman[52146]: 2024-09-10 04:02:41.110573671 +0000 UTC m=+0.570952026 container died ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T04:02:41.687 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T04:02:41.714 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 podman[52170]: 2024-09-10 04:02:41.662322247 +0000 UTC m=+0.545672596 container remove ba2db5c76dd387450f34b40d053097066ae782383d7964f845d19f93e8589433 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T04:02:41.714 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-10T04:02:41.714 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 systemd[1]: Stopped Ceph osd.5 for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:02:41.714 INFO:journalctl@ceph.osd.5.smithi050.stdout:Sep 10 04:02:41 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.5.service: Consumed 3.023s CPU time. 2024-09-10T04:02:41.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:41.732 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T04:02:41.732 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T04:02:41.732 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.6 2024-09-10T04:02:42.041 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:41 smithi050 systemd[1]: Stopping Ceph osd.6 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:42.041 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:41.901+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:42.041 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:41.901+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:42.041 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:41.901+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:42.041 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:41 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:41.901+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:42.304 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.040+0000 7f63bc738640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:42.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.040+0000 7f63bc738640 -1 osd.6 71 *** Got signal Terminated *** 2024-09-10T04:02:42.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.040+0000 7f63bc738640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:42.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.042+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:42.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.042+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:42.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:42.042+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:43.046+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:43.046+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:43.046+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:43.046+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:19.371797+0000 front 2024-09-10T04:02:19.371891+0000 (oldest deadline 2024-09-10T04:02:42.871620+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:42.878+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:42.878+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:43.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:42.878+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:43.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:42 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:42.878+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:44.084+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:44.084+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:44.084+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:44.084+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:19.371797+0000 front 2024-09-10T04:02:19.371891+0000 (oldest deadline 2024-09-10T04:02:42.871620+0000) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:43.852+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:44.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:43.852+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:44.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:43.852+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:44.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:43.852+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:44.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:43 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:43.852+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:45.107 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:44.824+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:45.107 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:44.824+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:45.107 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:44.824+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:45.107 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:44.824+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:45.107 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:44 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:44.824+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:45.555 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:45.106+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:45.555 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:45.106+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:45.555 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:45.106+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:45.555 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:45.106+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:19.371797+0000 front 2024-09-10T04:02:19.371891+0000 (oldest deadline 2024-09-10T04:02:42.871620+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:46.075+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:53.168352+0000 front 2024-09-10T04:01:53.168270+0000 (oldest deadline 2024-09-10T04:02:17.268064+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:46.075+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.069515+0000 front 2024-09-10T04:02:03.069590+0000 (oldest deadline 2024-09-10T04:02:26.569351+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:46.075+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:11.770837+0000 front 2024-09-10T04:02:11.770878+0000 (oldest deadline 2024-09-10T04:02:34.070811+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6[39380]: 2024-09-10T04:02:46.075+0000 7f63b8d4d640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:19.371797+0000 front 2024-09-10T04:02:19.371891+0000 (oldest deadline 2024-09-10T04:02:42.871620+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:45.838+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:46.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:45.838+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:46.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:45.838+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:46.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:45.838+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:46.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:45 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:45.838+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:47.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:46.868+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:47.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:46.868+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:47.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:46.868+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:47.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:46.868+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:47.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:46 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:46.868+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:47.842 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:47 smithi050 podman[52249]: 2024-09-10 04:02:47.579269977 +0000 UTC m=+5.619171113 container died c23f2e78118ee54871d49dcfbbb236c214dce7c5157fa135cb9d6fefa7aba0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T04:02:47.842 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:02:47.634Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:02:47.842 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:02:47.634Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nfs msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=nfs\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:02:47.842 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:02:47.635Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:02:47.842 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:02:47.636Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=smb msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=smb\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:02:47.842 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:02:47.637Z caller=refresh.go:90 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.38:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.38:8765: connect: connection refused" 2024-09-10T04:02:48.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52249]: 2024-09-10 04:02:48.065299161 +0000 UTC m=+6.105200298 container cleanup c23f2e78118ee54871d49dcfbbb236c214dce7c5157fa135cb9d6fefa7aba0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T04:02:48.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 bash[52249]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6 2024-09-10T04:02:48.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:47.906+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:48.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:47.906+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:48.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:47.906+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:48.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:47.906+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:48.306 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:47 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:47.907+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:48.697 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52267]: 2024-09-10 04:02:48.341694573 +0000 UTC m=+0.757601131 container remove c23f2e78118ee54871d49dcfbbb236c214dce7c5157fa135cb9d6fefa7aba0d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T04:02:48.948 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52353]: 2024-09-10 04:02:48.599357252 +0000 UTC m=+0.016229505 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:48.948 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52353]: 2024-09-10 04:02:48.709108189 +0000 UTC m=+0.125980485 container create 21c0e0eb067af4caff804745f5140361d029c2f1f7b7b20656e99d5ce61f7516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T04:02:48.948 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52353]: 2024-09-10 04:02:48.944706593 +0000 UTC m=+0.361578847 container init 21c0e0eb067af4caff804745f5140361d029c2f1f7b7b20656e99d5ce61f7516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:02:48.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:48.931+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:48.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:48.931+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:48.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:48.931+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:48.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:48.931+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:48.949 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:48 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:48.931+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:49.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:48 smithi050 podman[52353]: 2024-09-10 04:02:48.948089162 +0000 UTC m=+0.364961421 container start 21c0e0eb067af4caff804745f5140361d029c2f1f7b7b20656e99d5ce61f7516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6-deactivate, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T04:02:49.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:49 smithi050 podman[52353]: 2024-09-10 04:02:49.053557031 +0000 UTC m=+0.470429301 container attach 21c0e0eb067af4caff804745f5140361d029c2f1f7b7b20656e99d5ce61f7516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:02:49.305 INFO:journalctl@ceph.osd.6.smithi050.stdout:Sep 10 04:02:49 smithi050 podman[52353]: 2024-09-10 04:02:49.150103197 +0000 UTC m=+0.566975451 container died 21c0e0eb067af4caff804745f5140361d029c2f1f7b7b20656e99d5ce61f7516 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-6-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T04:02:49.791 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T04:02:49.831 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:49.831 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T04:02:49.832 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T04:02:49.832 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.7 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:49.903+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:49.903+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:49.903+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:49.903+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:49.903+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:50.055 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:49 smithi050 systemd[1]: Stopping Ceph osd.7 for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:51.197 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.870+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:51.197 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.870+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:51.197 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.870+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:51.198 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.870+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:51.198 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.871+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:51.198 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.965+0000 7f26ca668640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:51.198 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.965+0000 7f26ca668640 -1 osd.7 71 *** Got signal Terminated *** 2024-09-10T04:02:51.198 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:50 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:50.965+0000 7f26ca668640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T04:02:52.287 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:51 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:51.861+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:52.287 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:51 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:51.861+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:52.287 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:51 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:51.861+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:52.287 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:51 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:51.861+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:52.288 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:51 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:51.861+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:53.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:52 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:52.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:53.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:52 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:52.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:53.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:52 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:52.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:53.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:52 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:52.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:53.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:52 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:52.898+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:54.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:53 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:53.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:54.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:53 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:53.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:54.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:53 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:53.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:54.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:53 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:53.898+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:54.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:53 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:53.898+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-09-10T04:02:28.184169+0000 front 2024-09-10T04:02:28.184217+0000 (oldest deadline 2024-09-10T04:02:54.084249+0000) 2024-09-10T04:02:55.305 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:54 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:54.918+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-10T04:01:54.579897+0000 front 2024-09-10T04:01:54.579983+0000 (oldest deadline 2024-09-10T04:02:15.679726+0000) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6814 osd.1 since back 2024-09-10T04:02:03.781188+0000 front 2024-09-10T04:02:03.781315+0000 (oldest deadline 2024-09-10T04:02:29.081089+0000) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6822 osd.2 since back 2024-09-10T04:02:10.182071+0000 front 2024-09-10T04:02:10.181984+0000 (oldest deadline 2024-09-10T04:02:33.081761+0000) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.38:6830 osd.3 since back 2024-09-10T04:02:21.883069+0000 front 2024-09-10T04:02:21.883119+0000 (oldest deadline 2024-09-10T04:02:42.982876+0000) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.50:6804 osd.4 since back 2024-09-10T04:02:28.184169+0000 front 2024-09-10T04:02:28.184217+0000 (oldest deadline 2024-09-10T04:02:54.084249+0000) 2024-09-10T04:02:56.299 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:55 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7[42863]: 2024-09-10T04:02:55.960+0000 7f26c6c7d640 -1 osd.7 71 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.14574.0:92 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e71) 2024-09-10T04:02:56.554 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:56 smithi050 podman[52488]: 2024-09-10 04:02:56.297963862 +0000 UTC m=+5.853960672 container died 00103621eda53e89b00814ccb360665f923bc5c60eb0422d871425a805b86b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T04:02:56.918 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:56 smithi050 podman[52488]: 2024-09-10 04:02:56.696867416 +0000 UTC m=+6.252864222 container cleanup 00103621eda53e89b00814ccb360665f923bc5c60eb0422d871425a805b86b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2) 2024-09-10T04:02:56.918 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:56 smithi050 bash[52488]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7 2024-09-10T04:02:57.270 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:56 smithi050 podman[52611]: 2024-09-10 04:02:56.931168395 +0000 UTC m=+0.627332309 container remove 00103621eda53e89b00814ccb360665f923bc5c60eb0422d871425a805b86b82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T04:02:57.555 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.172476135 +0000 UTC m=+0.019433988 image pull d6d839bc8a4bdc14017733b053ef2e9b386c4d0f48b6d189c0888843abe03578 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca 2024-09-10T04:02:57.555 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.290774251 +0000 UTC m=+0.137732145 container create a5f0fd70e3d794214103f2548f30f4ecd1164cd0645e9f4598223a83f9fbfc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T04:02:57.555 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.501018369 +0000 UTC m=+0.347976212 container init a5f0fd70e3d794214103f2548f30f4ecd1164cd0645e9f4598223a83f9fbfc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T04:02:57.555 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.504479098 +0000 UTC m=+0.351436943 container start a5f0fd70e3d794214103f2548f30f4ecd1164cd0645e9f4598223a83f9fbfc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T04:02:57.996 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.584295212 +0000 UTC m=+0.431253055 container attach a5f0fd70e3d794214103f2548f30f4ecd1164cd0645e9f4598223a83f9fbfc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7-deactivate, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T04:02:57.996 INFO:journalctl@ceph.osd.7.smithi050.stdout:Sep 10 04:02:57 smithi050 podman[52831]: 2024-09-10 04:02:57.705821384 +0000 UTC m=+0.552779227 container died a5f0fd70e3d794214103f2548f30f4ecd1164cd0645e9f4598223a83f9fbfc04 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1) 2024-09-10T04:02:58.319 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T04:02:58.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:02:58.365 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T04:02:58.365 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T04:02:58.365 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T04:02:58.693 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:02:58 smithi038 systemd[1]: Stopping Ceph rgw.foo.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:02:59.004 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:02:58 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a[48463]: 2024-09-10T04:02:58.691+0000 7f8d5024d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T04:02:59.004 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:02:58 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a[48463]: 2024-09-10T04:02:58.691+0000 7f8d53ab9880 -1 shutting down 2024-09-10T04:03:08.994 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:03:08 smithi038 bash[61876]: time="2024-09-10T04:03:08Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T04:03:08.995 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:03:08 smithi038 podman[61876]: 2024-09-10 04:03:08.716496098 +0000 UTC m=+10.141118598 container died 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T04:03:09.254 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:03:09 smithi038 podman[61876]: 2024-09-10 04:03:09.116879494 +0000 UTC m=+10.541501986 container cleanup 8d456acce75da78fed2bbfd86b67707970c749815be12f42afe607c676343781 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux ) 2024-09-10T04:03:09.254 INFO:journalctl@ceph.rgw.foo.a.smithi038.stdout:Sep 10 04:03:09 smithi038 bash[61876]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T04:03:09.481 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T04:03:09.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:03:09.520 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T04:03:09.520 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T04:03:09.520 DEBUG:teuthology.orchestra.run.smithi050:> sudo systemctl stop ceph-3be9daae-6f28-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T04:03:09.805 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 systemd[1]: Stopping Ceph prometheus.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.868Z caller=main.go:964 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.868Z caller=main.go:988 level=info msg="Stopping scrape discovery manager..." 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.868Z caller=main.go:1002 level=info msg="Stopping notify discovery manager..." 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.868Z caller=manager.go:177 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:984 level=info msg="Scrape discovery manager stopped" 2024-09-10T04:03:10.137 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:998 level=info msg="Notify discovery manager stopped" 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=manager.go:187 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:1039 level=info msg="Stopping scrape manager..." 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:1031 level=info msg="Scrape manager stopped" 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=notifier.go:618 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:1261 level=info msg="Notifier manager stopped" 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a[46599]: ts=2024-09-10T04:03:09.869Z caller=main.go:1273 level=info msg="See you next time!" 2024-09-10T04:03:10.138 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:09 smithi050 podman[53456]: 2024-09-10 04:03:09.87931735 +0000 UTC m=+0.149173808 container died 2ebfb9ed5d9c6980e071762f0e1d760d576ef7562990555a6331a68386d52e52 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:10.487 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:10 smithi050 podman[53456]: 2024-09-10 04:03:10.302352121 +0000 UTC m=+0.572208578 container cleanup 2ebfb9ed5d9c6980e071762f0e1d760d576ef7562990555a6331a68386d52e52 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:10.487 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:10 smithi050 bash[53456]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T04:03:10.804 INFO:journalctl@ceph.prometheus.a.smithi050.stdout:Sep 10 04:03:10 smithi050 podman[53472]: 2024-09-10 04:03:10.486952911 +0000 UTC m=+0.605534807 container remove 2ebfb9ed5d9c6980e071762f0e1d760d576ef7562990555a6331a68386d52e52 (image=quay.io/prometheus/prometheus:v2.51.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:11.692 DEBUG:teuthology.orchestra.run.smithi050:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3be9daae-6f28-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T04:03:11.732 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T04:03:11.732 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T04:03:11.732 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T04:03:11.946 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T04:03:14.240 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:13 smithi038 systemd[1]: Stopping Ceph alertmanager.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:14.504 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:14 smithi038 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a[55407]: ts=2024-09-10T04:03:14.239Z caller=main.go:542 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T04:03:14.504 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:14 smithi038 podman[62979]: 2024-09-10 04:03:14.259544813 +0000 UTC m=+0.122953900 container died 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:14.832 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:14 smithi038 podman[62979]: 2024-09-10 04:03:14.832023319 +0000 UTC m=+0.695432409 container cleanup 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:15.117 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:14 smithi038 bash[62979]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T04:03:15.117 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:15 smithi038 podman[62991]: 2024-09-10 04:03:15.058186739 +0000 UTC m=+0.794405640 container remove 97455da53b47a8336777399452bec46360e2c85c16afc3633af1aad39a43ec52 (image=quay.io/prometheus/alertmanager:v0.27.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:15.117 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:15 smithi038 podman[62991]: 2024-09-10 04:03:15.117005392 +0000 UTC m=+0.853224293 volume remove e84156c85523826948b545bfe949430bf8260aaff3910ff211655c54f23f864f 2024-09-10T04:03:15.465 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:15 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T04:03:15.465 INFO:journalctl@ceph.alertmanager.a.smithi038.stdout:Sep 10 04:03:15 smithi038 systemd[1]: Stopped Ceph alertmanager.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:03:15.754 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:15 smithi038 systemd[1]: Stopping Ceph node-exporter.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:15.754 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:15 smithi038 podman[63105]: 2024-09-10 04:03:15.714568416 +0000 UTC m=+0.106485804 container died db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:16.254 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 podman[63105]: 2024-09-10 04:03:16.037360349 +0000 UTC m=+0.429277738 container cleanup db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:16.254 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 bash[63105]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T04:03:16.254 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 podman[63117]: 2024-09-10 04:03:16.188389144 +0000 UTC m=+0.477531738 container remove db0d46cc90a08c668cebf70c2538489828d38c1f27f0877d8251245508a638d2 (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T04:03:16.255 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T04:03:16.716 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T04:03:16.717 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 systemd[1]: Stopped Ceph node-exporter.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:03:16.717 INFO:journalctl@ceph.node-exporter.a.smithi038.stdout:Sep 10 04:03:16 smithi038 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 1.830s CPU time. 2024-09-10T04:03:17.123 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T04:03:17.337 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T04:03:19.055 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:18 smithi050 systemd[1]: Stopping Ceph iscsi.iscsi.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:19.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:19 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a[45233]: debug Shutdown received 2024-09-10T04:03:29.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:29 smithi050 bash[54058]: time="2024-09-10T04:03:29Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T04:03:29.555 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:29 smithi050 podman[54058]: 2024-09-10 04:03:29.266857086 +0000 UTC m=+10.127113372 container died 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T04:03:30.012 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:29 smithi050 podman[54058]: 2024-09-10 04:03:29.843621687 +0000 UTC m=+10.703877950 container cleanup 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, ceph=True) 2024-09-10T04:03:30.012 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:29 smithi050 bash[54058]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T04:03:30.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:30 smithi050 podman[54071]: 2024-09-10 04:03:30.011705065 +0000 UTC m=+0.743141621 container remove 53bc3ecbe2577e5c8591a462103847fde157d5555afbc2f47ab0c902f10c50c6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2db6858ffc169715474ea237dd8b2beae440a5514686b8202506e12cb7dc1fca, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-adk-testing-2024-09-09-1904-3a5e3f1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T04:03:30.305 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:30 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T04:03:30.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:30 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T04:03:30.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:30 smithi050 systemd[1]: Stopped Ceph iscsi.iscsi.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:03:30.306 INFO:journalctl@ceph.iscsi.iscsi.a.smithi050.stdout:Sep 10 04:03:30 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.000s CPU time. 2024-09-10T04:03:32.027 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:31 smithi050 systemd[1]: Stopping Ceph grafana.a for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=server t=2024-09-10T04:03:32.02704439Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=tracing t=2024-09-10T04:03:32.027223465Z level=info msg="Closing tracing" 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=ticker t=2024-09-10T04:03:32.027240834Z level=info msg=stopped last_tick=2024-09-10T04:03:30Z 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=grafana-apiserver t=2024-09-10T04:03:32.027284411Z level=info msg="StorageObjectCountTracker pruner is exiting" 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a[48585]: logger=sqlstore.transactions t=2024-09-10T04:03:32.038228794Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T04:03:32.305 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 podman[54333]: 2024-09-10 04:03:32.047899405 +0000 UTC m=+0.115605811 container died 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:03:32.805 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 podman[54333]: 2024-09-10 04:03:32.660160673 +0000 UTC m=+0.727867069 container cleanup 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:03:32.805 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 bash[54333]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a 2024-09-10T04:03:33.101 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:32 smithi050 podman[54345]: 2024-09-10 04:03:32.836638918 +0000 UTC m=+0.785140899 container remove 7a08e9c9edb032a240b1437ca3b111f8683453cdc8eb017c415a66c16ef96288 (image=quay.io/ceph/grafana:10.4.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-grafana-a, maintainer=Grafana Labs ) 2024-09-10T04:03:33.102 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:33 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@grafana.a.service: Deactivated successfully. 2024-09-10T04:03:33.102 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:33 smithi050 systemd[1]: Stopped Ceph grafana.a for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:03:33.102 INFO:journalctl@ceph.grafana.a.smithi050.stdout:Sep 10 04:03:33 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@grafana.a.service: Consumed 4.225s CPU time. 2024-09-10T04:03:33.550 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:33 smithi050 systemd[1]: Stopping Ceph node-exporter.b for 3be9daae-6f28-11ef-bcea-c7b262605968... 2024-09-10T04:03:33.804 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:33 smithi050 podman[54458]: 2024-09-10 04:03:33.550525551 +0000 UTC m=+0.109136247 container died 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:03:34.234 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:33 smithi050 podman[54458]: 2024-09-10 04:03:33.881183485 +0000 UTC m=+0.439794180 container cleanup 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:03:34.234 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:33 smithi050 bash[54458]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T04:03:34.234 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:34 smithi050 podman[54470]: 2024-09-10 04:03:34.032181946 +0000 UTC m=+0.485564703 container remove 98ba52dc14b1f564e500c927b50d08631e602abd1d12df71a91f90ca480bad1e (image=quay.io/prometheus/node-exporter:v1.7.0, name=ceph-3be9daae-6f28-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T04:03:34.234 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:34 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T04:03:34.519 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:34 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T04:03:34.519 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:34 smithi050 systemd[1]: Stopped Ceph node-exporter.b for 3be9daae-6f28-11ef-bcea-c7b262605968. 2024-09-10T04:03:34.519 INFO:journalctl@ceph.node-exporter.b.smithi050.stdout:Sep 10 04:03:34 smithi050 systemd[1]: ceph-3be9daae-6f28-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 1.741s CPU time. 2024-09-10T04:03:34.994 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:03:35.029 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T04:03:35.059 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T04:03:35.061 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi038/crash 2024-09-10T04:03:35.062 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash -- . 2024-09-10T04:03:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T04:03:35.111 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-09-10T04:03:35.113 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi050/crash 2024-09-10T04:03:35.114 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash -- . 2024-09-10T04:03:35.155 INFO:teuthology.orchestra.run.smithi050.stderr:tar: /var/lib/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T04:03:35.155 INFO:teuthology.orchestra.run.smithi050.stderr:tar: Error is not recoverable: exiting now 2024-09-10T04:03:35.156 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T04:03:35.156 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v CEPHADM_AGENT_DOWN | head -n 1 2024-09-10T04:03:35.190 INFO:tasks.cephadm:Compressing logs... 2024-09-10T04:03:35.190 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T04:03:35.234 DEBUG:teuthology.orchestra.run.smithi050:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T04:03:35.267 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/tcmu-runner.log: /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.7.log: 63.5% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log 2024-09-10T04:03:35.268 INFO:teuthology.orchestra.run.smithi050.stderr: 79.2% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.x.log: 90.3% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T04:03:35.269 INFO:teuthology.orchestra.run.smithi050.stderr: 85.7% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T04:03:35.273 INFO:teuthology.orchestra.run.smithi050.stderr: 89.9% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T04:03:35.273 INFO:teuthology.orchestra.run.smithi050.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T04:03:35.277 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T04:03:35.278 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr: 89.0% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log: /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.y.log: 93.8% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T04:03:35.279 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T04:03:35.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log: 91.6% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T04:03:35.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T04:03:35.280 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.9% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T04:03:35.280 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T04:03:35.286 INFO:teuthology.orchestra.run.smithi050.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T04:03:35.300 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 87.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T04:03:35.311 INFO:teuthology.orchestra.run.smithi050.stderr: 90.2% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T04:03:35.317 INFO:teuthology.orchestra.run.smithi038.stderr: 90.0% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T04:03:35.399 INFO:teuthology.orchestra.run.smithi038.stderr: 93.0% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T04:03:35.401 INFO:teuthology.orchestra.run.smithi050.stderr: 93.1% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T04:03:35.416 INFO:teuthology.orchestra.run.smithi038.stderr: 92.3% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T04:03:35.422 INFO:teuthology.orchestra.run.smithi050.stderr: 93.2% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T04:03:35.422 INFO:teuthology.orchestra.run.smithi050.stderr: 93.1% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T04:03:35.430 INFO:teuthology.orchestra.run.smithi050.stderr: 92.1% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T04:03:35.434 INFO:teuthology.orchestra.run.smithi038.stderr: 88.4% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T04:03:35.495 INFO:teuthology.orchestra.run.smithi038.stderr: 93.4% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T04:03:35.511 INFO:teuthology.orchestra.run.smithi050.stderr: 93.5% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T04:03:35.513 INFO:teuthology.orchestra.run.smithi050.stderr: 2024-09-10T04:03:35.513 INFO:teuthology.orchestra.run.smithi050.stderr:real 0m0.264s 2024-09-10T04:03:35.513 INFO:teuthology.orchestra.run.smithi050.stderr:user 0m0.933s 2024-09-10T04:03:35.513 INFO:teuthology.orchestra.run.smithi050.stderr:sys 0m0.045s 2024-09-10T04:03:35.525 INFO:teuthology.orchestra.run.smithi038.stderr: 93.5% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T04:03:35.551 INFO:teuthology.orchestra.run.smithi038.stderr: 93.6% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T04:03:35.630 INFO:teuthology.orchestra.run.smithi038.stderr: 91.2% -- replaced with /var/log/ceph/3be9daae-6f28-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T04:03:35.632 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-10T04:03:35.632 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m0.384s 2024-09-10T04:03:35.632 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m1.634s 2024-09-10T04:03:35.632 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.065s 2024-09-10T04:03:35.633 INFO:tasks.cephadm:Archiving logs... 2024-09-10T04:03:35.633 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi038/log 2024-09-10T04:03:35.634 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T04:03:35.905 DEBUG:teuthology.misc:Transferring archived files from smithi050:/var/log/ceph to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi050/log 2024-09-10T04:03:35.905 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T04:03:36.056 INFO:tasks.cephadm:Removing cluster... 2024-09-10T04:03:36.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 --force 2024-09-10T04:03:36.256 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T04:03:36.377 DEBUG:teuthology.orchestra.run.smithi050:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3be9daae-6f28-11ef-bcea-c7b262605968 --force 2024-09-10T04:03:36.581 INFO:teuthology.orchestra.run.smithi050.stdout:Deleting cluster with fsid: 3be9daae-6f28-11ef-bcea-c7b262605968 2024-09-10T04:03:36.704 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T04:03:36.705 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T04:03:36.721 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T04:03:36.737 INFO:tasks.cephadm:Teardown complete 2024-09-10T04:03:36.738 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-10T04:03:36.752 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi038:/dev/vg_nvme/lv_1... 2024-09-10T04:03:36.752 DEBUG:teuthology.orchestra.run.smithi038:> sudo nvme disconnect -n lv_1 2024-09-10T04:03:36.909 INFO:teuthology.orchestra.run.smithi038.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-10T04:03:36.911 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi038:/dev/vg_nvme/lv_2... 2024-09-10T04:03:36.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo nvme disconnect -n lv_2 2024-09-10T04:03:37.055 INFO:teuthology.orchestra.run.smithi038.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-10T04:03:37.056 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi038:/dev/vg_nvme/lv_3... 2024-09-10T04:03:37.057 DEBUG:teuthology.orchestra.run.smithi038:> sudo nvme disconnect -n lv_3 2024-09-10T04:03:37.225 INFO:teuthology.orchestra.run.smithi038.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-10T04:03:37.226 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi038:/dev/vg_nvme/lv_4... 2024-09-10T04:03:37.226 DEBUG:teuthology.orchestra.run.smithi038:> sudo nvme disconnect -n lv_4 2024-09-10T04:03:37.381 INFO:teuthology.orchestra.run.smithi038.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-10T04:03:37.382 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-10T04:03:37.382 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/scratch_devs 2024-09-10T04:03:37.408 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi050:/dev/vg_nvme/lv_1... 2024-09-10T04:03:37.409 DEBUG:teuthology.orchestra.run.smithi050:> sudo nvme disconnect -n lv_1 2024-09-10T04:03:37.574 INFO:teuthology.orchestra.run.smithi050.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-10T04:03:37.576 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi050:/dev/vg_nvme/lv_2... 2024-09-10T04:03:37.576 DEBUG:teuthology.orchestra.run.smithi050:> sudo nvme disconnect -n lv_2 2024-09-10T04:03:37.745 INFO:teuthology.orchestra.run.smithi050.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-10T04:03:37.749 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi050:/dev/vg_nvme/lv_3... 2024-09-10T04:03:37.749 DEBUG:teuthology.orchestra.run.smithi050:> sudo nvme disconnect -n lv_3 2024-09-10T04:03:37.907 INFO:teuthology.orchestra.run.smithi050.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-10T04:03:37.909 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi050:/dev/vg_nvme/lv_4... 2024-09-10T04:03:37.909 DEBUG:teuthology.orchestra.run.smithi050:> sudo nvme disconnect -n lv_4 2024-09-10T04:03:38.055 INFO:teuthology.orchestra.run.smithi050.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-10T04:03:38.057 DEBUG:teuthology.orchestra.run.smithi050:> set -ex 2024-09-10T04:03:38.057 DEBUG:teuthology.orchestra.run.smithi050:> sudo dd of=/scratch_devs 2024-09-10T04:03:38.088 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T04:03:38.096 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T04:03:38.096 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T04:03:38.099 DEBUG:teuthology.orchestra.run.smithi050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T04:03:38.111 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 35 -37us[ -26us] +/- 37ms 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -443us[ -432us] +/- 52ms 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 +754us[ +754us] +/- 77ms 2024-09-10T04:03:38.116 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T04:03:38.146 INFO:teuthology.orchestra.run.smithi050.stderr:bash: line 1: ntpq: command not found 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:=============================================================================== 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 34 -32us[ -32us] +/- 37ms 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -446us[ -445us] +/- 52ms 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 +771us[ +772us] +/- 77ms 2024-09-10T04:03:38.152 INFO:teuthology.orchestra.run.smithi050.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T04:03:38.153 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T04:03:38.165 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T04:03:38.166 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T04:03:38.175 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T04:03:38.244 DEBUG:teuthology.orchestra.run.smithi050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T04:03:38.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T04:03:38.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:03:38.350 DEBUG:teuthology.orchestra.run.smithi050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T04:03:38.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:03:38.381 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T04:03:38.391 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T04:03:38.400 INFO:teuthology.task.internal:Duration was 857.583138 seconds 2024-09-10T04:03:38.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T04:03:38.409 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T04:03:38.409 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T04:03:38.412 DEBUG:teuthology.orchestra.run.smithi050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T04:03:38.450 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T04:03:38.464 INFO:teuthology.orchestra.run.smithi050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T04:03:38.852 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T04:03:38.852 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-09-10T04:03:38.852 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T04:03:38.875 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi050.front.sepia.ceph.com 2024-09-10T04:03:38.875 DEBUG:teuthology.orchestra.run.smithi050:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T04:03:38.914 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T04:03:38.914 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T04:03:38.917 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T04:03:38.984 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T04:03:38.985 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T04:03:38.987 DEBUG:teuthology.orchestra.run.smithi050:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T04:03:39.244 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T04:03:39.255 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T04:03:39.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T04:03:39.282 DEBUG:teuthology.orchestra.run.smithi050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T04:03:39.308 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T04:03:39.325 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:03:39.328 DEBUG:teuthology.orchestra.run.smithi050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:03:39.349 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-09-10T04:03:39.372 INFO:teuthology.orchestra.run.smithi050.stdout:kernel.core_pattern = core 2024-09-10T04:03:39.396 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:03:39.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:03:39.431 DEBUG:teuthology.orchestra.run.smithi050:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T04:03:39.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T04:03:39.452 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T04:03:39.463 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T04:03:39.464 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi038 2024-09-10T04:03:39.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T04:03:39.512 DEBUG:teuthology.misc:Transferring archived files from smithi050:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/adking-2024-09-10_02:59:23-orch:cephadm-wip-adk-testing-2024-09-09-1904-distro-default-smithi/7898607/remote/smithi050 2024-09-10T04:03:39.513 DEBUG:teuthology.orchestra.run.smithi050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T04:03:39.550 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T04:03:39.550 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T04:03:39.553 DEBUG:teuthology.orchestra.run.smithi050:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T04:03:39.600 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T04:03:39.610 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T04:03:39.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T04:03:39.619 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T04:03:39.619 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T04:03:39.622 DEBUG:teuthology.orchestra.run.smithi050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T04:03:39.635 INFO:teuthology.orchestra.run.smithi038.stdout: 265861 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 04:03 /home/ubuntu/cephtest 2024-09-10T04:03:39.656 INFO:teuthology.orchestra.run.smithi050.stdout: 265613 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 04:03 /home/ubuntu/cephtest 2024-09-10T04:03:39.657 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T04:03:39.667 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T04:03:39.688 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} duration: 857.5831377506256 owner: scheduled_adking@teuthology success: true 2024-09-10T04:03:39.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T04:03:39.782 INFO:teuthology.run:pass