2024-09-10T18:23:22.289 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:23:22.304 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:23:22.381 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} email: gabrioux@ibm.com first_in_suite: false job_id: '7900234' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: true mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - MON_DOWN - mons down - mon down - out of quorum sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 kernel: hwe: true selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi066.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGN58QItvrfE8+4ujkKZcmQ1FJx5O9j822QDkpMTWiOzMPLkZyMlmILoDsOTl2Afb9ovboPGB8Sklf/JNyccIMA= smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM3hl/nU4euqbrTNmMBIDV03DKLHGu0Ae5nHQPRmso+eo+SHjFcbSZezngvbI6wdKj4D2vqeXaxuAczBPYDMYlg= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:23:22.381 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:23:22.382 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:23:22.382 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:23:22.383 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:23:22.404 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:23:22.404 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:23:22.404 INFO:teuthology.packaging:ref: None 2024-09-10T18:23:22.404 INFO:teuthology.packaging:tag: None 2024-09-10T18:23:22.404 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:23:22.404 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:23:22.404 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:23:22.547 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:23:22.548 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:23:22.557 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:23:22.557 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:23:22.566 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:23:22.576 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:23:22.584 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:23:22.609 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi066.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:17:39.190970', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGN58QItvrfE8+4ujkKZcmQ1FJx5O9j822QDkpMTWiOzMPLkZyMlmILoDsOTl2Afb9ovboPGB8Sklf/JNyccIMA='} 2024-09-10T18:23:22.630 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:17:39.189784', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM3hl/nU4euqbrTNmMBIDV03DKLHGu0Ae5nHQPRmso+eo+SHjFcbSZezngvbI6wdKj4D2vqeXaxuAczBPYDMYlg='} 2024-09-10T18:23:22.630 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:23:22.638 INFO:teuthology.task.internal:roles: ubuntu@smithi066.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:23:22.638 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:23:22.638 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:23:22.701 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1b75d8bac0>, signals=[15]) 2024-09-10T18:23:22.701 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:23:22.709 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:23:22.709 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T18:23:22.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:23:22.786 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T18:23:22.786 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:23:22.862 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:23:22.869 DEBUG:teuthology.orchestra.run.smithi066:> uname -m 2024-09-10T18:23:22.889 INFO:teuthology.orchestra.run.smithi066.stdout:x86_64 2024-09-10T18:23:22.889 DEBUG:teuthology.orchestra.run.smithi066:> cat /etc/os-release 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:NAME="CentOS Stream" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION="9" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:ID="centos" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:VERSION_ID="9" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:ANSI_COLOR="0;31" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:23:22.948 INFO:teuthology.orchestra.run.smithi066.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:23:22.949 INFO:teuthology.orchestra.run.smithi066.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:23:22.949 INFO:teuthology.orchestra.run.smithi066.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:23:22.949 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:23:22.949 INFO:teuthology.orchestra.run.smithi066.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:23:22.949 INFO:teuthology.lock.ops:Updating smithi066.front.sepia.ceph.com on lock server 2024-09-10T18:23:22.968 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-09-10T18:23:22.986 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-09-10T18:23:22.986 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="CentOS Stream" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="9" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:ID="centos" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="9" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:ANSI_COLOR="0;31" 2024-09-10T18:23:23.045 INFO:teuthology.orchestra.run.smithi117.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:23:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:23:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:23:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:23:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:23:23.046 INFO:teuthology.orchestra.run.smithi117.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:23:23.046 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-09-10T18:23:23.068 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:23:23.077 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:23:23.085 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:23:23.085 DEBUG:teuthology.orchestra.run.smithi066:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:23:23.087 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:23:23.104 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:23:23.112 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:23:23.112 DEBUG:teuthology.orchestra.run.smithi066:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:23:23.145 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:23:23.175 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:23:23.224 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:23:23.239 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:23:23.239 INFO:teuthology.task.kernel:normalize config orig: {'hwe': True} 2024-09-10T18:23:23.239 DEBUG:teuthology.task.kernel:normalized overrides {'mon.a': {'hwe': True}, 'mon.c': {'hwe': True}, 'mgr.y': {'hwe': True}, 'osd.0': {'hwe': True}, 'osd.1': {'hwe': True}, 'osd.2': {'hwe': True}, 'osd.3': {'hwe': True}, 'client.0': {'hwe': True}, 'ceph.rgw.foo.a': {'hwe': True}, 'node-exporter.a': {'hwe': True}, 'alertmanager.a': {'hwe': True}, 'mon.b': {'hwe': True}, 'mgr.x': {'hwe': True}, 'osd.4': {'hwe': True}, 'osd.5': {'hwe': True}, 'osd.6': {'hwe': True}, 'osd.7': {'hwe': True}, 'client.1': {'hwe': True}, 'prometheus.a': {'hwe': True}, 'grafana.a': {'hwe': True}, 'node-exporter.b': {'hwe': True}, 'ceph.iscsi.iscsi.a': {'hwe': True}} 2024-09-10T18:23:23.239 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro', 'hwe': True}, 'mon.b': {'kdb': 1, 'sha1': 'distro', 'hwe': True}}, timeout 300 2024-09-10T18:23:23.239 DEBUG:teuthology.orchestra.run.smithi066:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:23:23.240 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:23:23.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:23:23.257 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-10T18:23:23.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:23:23.258 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T18:23:23.315 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:23:23.315 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.14.0-503.el9.x86_64 2024-09-10T18:23:23.316 DEBUG:teuthology.orchestra.run.smithi117:> sudo yum install -y kernel 2024-09-10T18:23:23.317 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:23:23.317 INFO:teuthology.task.kernel:Running kernel on smithi066: 5.14.0-503.el9.x86_64 2024-09-10T18:23:23.317 DEBUG:teuthology.orchestra.run.smithi066:> sudo yum install -y kernel 2024-09-10T18:23:25.712 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.2 MB 00:00 2024-09-10T18:23:25.775 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T18:23:27.768 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-10T18:23:30.155 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - AppStream 6.2 MB/s | 20 MB 00:03 2024-09-10T18:23:32.385 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - CRB 8.1 MB/s | 6.5 MB 00:00 2024-09-10T18:23:34.043 INFO:teuthology.orchestra.run.smithi117.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 19 kB 00:00 2024-09-10T18:23:34.822 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-10T18:23:34.939 INFO:teuthology.orchestra.run.smithi117.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-10T18:23:36.431 INFO:teuthology.orchestra.run.smithi066.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-10T18:23:37.319 INFO:teuthology.orchestra.run.smithi066.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:23:40.445 INFO:teuthology.orchestra.run.smithi117.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:23:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:23:42.418 INFO:teuthology.orchestra.run.smithi117.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:23:42.508 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-10T18:23:42.519 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:23:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-10T18:23:42.519 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:23:42.519 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-10T18:23:42.519 INFO:teuthology.orchestra.run.smithi117.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout:Install 4 Packages 2024-09-10T18:23:42.520 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:23:42.521 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 79 M 2024-09-10T18:23:42.521 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 126 M 2024-09-10T18:23:42.521 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-10T18:23:42.921 INFO:teuthology.orchestra.run.smithi117.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 188 kB/s | 23 kB 00:00 2024-09-10T18:23:42.932 INFO:teuthology.orchestra.run.smithi066.stdout:lab-extras 48 kB/s | 1.7 kB 00:00 2024-09-10T18:23:43.821 INFO:teuthology.orchestra.run.smithi117.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-10T18:23:44.521 INFO:teuthology.orchestra.run.smithi117.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 20 MB/s | 35 MB 00:01 2024-09-10T18:23:44.997 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:23:44.997 INFO:teuthology.orchestra.run.smithi066.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:23:45.046 INFO:teuthology.orchestra.run.smithi117.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 13 MB/s | 29 MB 00:02 2024-09-10T18:23:45.046 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:23:45.047 INFO:teuthology.orchestra.run.smithi117.stdout:Total 31 MB/s | 79 MB 00:02 2024-09-10T18:23:45.085 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-09-10T18:23:45.094 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout:Install 4 Packages 2024-09-10T18:23:45.095 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:23:45.096 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 79 M 2024-09-10T18:23:45.096 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 126 M 2024-09-10T18:23:45.096 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-09-10T18:23:45.401 INFO:teuthology.orchestra.run.smithi066.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 80 kB/s | 23 kB 00:00 2024-09-10T18:23:45.725 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-10T18:23:46.068 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-10T18:23:46.068 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-10T18:23:47.009 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-10T18:23:47.009 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-10T18:23:48.334 INFO:teuthology.orchestra.run.smithi066.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 11 MB/s | 35 MB 00:03 2024-09-10T18:23:49.455 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-10T18:23:50.157 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:23:50.248 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:53.021 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:23:53.149 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:57.966 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:23:58.057 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:02.533 INFO:teuthology.orchestra.run.smithi066.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 920 kB/s | 16 MB 00:17 2024-09-10T18:24:02.889 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:03.742 INFO:teuthology.orchestra.run.smithi066.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 1.6 MB/s | 29 MB 00:18 2024-09-10T18:24:03.742 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:24:03.742 INFO:teuthology.orchestra.run.smithi066.stdout:Total 4.2 MB/s | 79 MB 00:18 2024-09-10T18:24:04.457 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-10T18:24:05.008 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-10T18:24:05.008 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-10T18:24:06.016 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-10T18:24:06.016 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-10T18:24:08.622 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-10T18:24:09.301 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:24:09.384 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:24:12.323 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:24:12.452 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:24:17.673 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:24:17.719 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:22.967 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:40.942 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:40.982 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:24:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:24:41.331 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:24:42.219 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:24:42.219 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:24:42.219 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-10T18:24:42.219 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.219 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.220 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.220 INFO:teuthology.orchestra.run.smithi117.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.220 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:24:42.220 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-10T18:24:42.566 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:42.654 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.655 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:42.655 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:24:42.655 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:24:42.655 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:24:42.655 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:24:42.655 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:24:42.655 DEBUG:teuthology.orchestra.run.smithi117:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:24:42.714 INFO:teuthology.orchestra.run.smithi117.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:24:42.714 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:24:42.714 DEBUG:teuthology.orchestra.run.smithi117:> sudo rpm -qi grub2-tools 2024-09-10T18:24:42.752 INFO:teuthology.orchestra.run.smithi117.stdout:Name : grub2-tools 2024-09-10T18:24:42.752 INFO:teuthology.orchestra.run.smithi117.stdout:Epoch : 1 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Version : 2.06 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Release : 82.el9 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: x86_64 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Group : Unspecified 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Size : 8274670 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:License : GPLv3+ 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:24:42.753 INFO:teuthology.orchestra.run.smithi117.stdout:Packager : builder@centos.org 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:Vendor : CentOS 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:Summary : Support tools for GRUB. 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:Description : 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:hardware devices. 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:24:42.754 INFO:teuthology.orchestra.run.smithi117.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:24:42.755 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:24:42.755 INFO:teuthology.task.kernel:Updating grub on smithi117 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:24:42.755 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:24:43.393 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-09-10T18:24:44.699 INFO:teuthology.orchestra.run.smithi117.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:24:44.725 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-09-10T18:24:44.727 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-09-10T18:24:44.742 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.Yo2evHS8bT 2024-09-10T18:24:44.742 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Yo2evHS8bT 2024-09-10T18:24:44.808 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.Yo2evHS8bT 2024-09-10T18:24:44.947 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.Yo2evHS8bT is 6KB 2024-09-10T18:24:44.995 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.Yo2evHS8bT 2024-09-10T18:24:45.010 DEBUG:teuthology.orchestra.run.smithi117:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:24:45.073 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:24:45.073 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:24:45.073 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:24:45.073 INFO:teuthology.orchestra.run.smithi117.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:24:45.074 DEBUG:teuthology.orchestra.run.smithi117:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:24:45.220 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-09-10T18:25:02.951 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:25:02.992 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:25:03.294 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:25:03.295 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:25:03.295 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:25:03.295 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:25:04.137 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-10T18:25:04.495 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:25:04.585 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.585 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:25:04.586 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:25:04.586 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:25:04.586 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:25:04.586 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:25:04.586 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi066.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:25:04.586 DEBUG:teuthology.orchestra.run.smithi066:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:25:04.647 INFO:teuthology.orchestra.run.smithi066.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:25:04.648 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:25:04.648 DEBUG:teuthology.orchestra.run.smithi066:> sudo rpm -qi grub2-tools 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Name : grub2-tools 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Epoch : 1 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Version : 2.06 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Release : 82.el9 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Architecture: x86_64 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Group : Unspecified 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:Size : 8274670 2024-09-10T18:25:04.688 INFO:teuthology.orchestra.run.smithi066.stdout:License : GPLv3+ 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Packager : builder@centos.org 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Vendor : CentOS 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Summary : Support tools for GRUB. 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:Description : 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:25:04.689 INFO:teuthology.orchestra.run.smithi066.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:25:04.690 INFO:teuthology.orchestra.run.smithi066.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:25:04.690 INFO:teuthology.orchestra.run.smithi066.stdout:hardware devices. 2024-09-10T18:25:04.690 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:25:04.690 INFO:teuthology.orchestra.run.smithi066.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:25:04.691 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:25:04.691 INFO:teuthology.task.kernel:Updating grub on smithi066 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:25:04.691 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:25:05.380 INFO:teuthology.orchestra.run.smithi066.stderr:Generating grub configuration file ... 2024-09-10T18:25:06.708 INFO:teuthology.orchestra.run.smithi066.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:25:06.747 INFO:teuthology.orchestra.run.smithi066.stderr:done 2024-09-10T18:25:06.749 DEBUG:teuthology.orchestra.run.smithi066:> mktemp 2024-09-10T18:25:06.765 INFO:teuthology.orchestra.run.smithi066.stdout:/tmp/tmp.qGAoMEUx9c 2024-09-10T18:25:06.765 DEBUG:teuthology.orchestra.run.smithi066:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.qGAoMEUx9c 2024-09-10T18:25:06.832 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0666 /tmp/tmp.qGAoMEUx9c 2024-09-10T18:25:06.973 DEBUG:teuthology.orchestra.remote:smithi066:/tmp/tmp.qGAoMEUx9c is 6KB 2024-09-10T18:25:07.021 DEBUG:teuthology.orchestra.run.smithi066:> rm -fr /tmp/tmp.qGAoMEUx9c 2024-09-10T18:25:07.036 DEBUG:teuthology.orchestra.run.smithi066:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:25:07.100 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:25:07.100 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:25:07.101 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:25:07.101 INFO:teuthology.orchestra.run.smithi066.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:25:07.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:25:07.263 DEBUG:teuthology.orchestra.run.smithi066:> sudo shutdown -r now 2024-09-10T18:25:15.229 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:25:15.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T18:25:15.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:25:37.285 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:25:37.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:25:37.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:11.901 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T18:26:15.233 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:26:20.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:26:20.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:23.965 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.66 2024-09-10T18:26:24.234 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T18:26:24.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:26:24.582 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-10T18:26:25.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T18:26:25.039 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:26:25.039 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-09-10T18:26:25.057 INFO:teuthology.orchestra.run.smithi117.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:26:25.057 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:26:25.057 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:26:25.058 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:26:26.058 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:26:26.059 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:26:26.187 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-09-10T18:26:26.206 DEBUG:teuthology.parallel:result is None 2024-09-10T18:26:35.966 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:26:35.967 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:27:35.970 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T18:27:50.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:27:50.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:27:51.317 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T18:27:51.672 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:27:51.672 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:27:51.672 DEBUG:teuthology.orchestra.run.smithi066:> uname -r 2024-09-10T18:27:51.688 INFO:teuthology.orchestra.run.smithi066.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:27:51.692 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:27:51.692 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:27:51.692 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:27:52.693 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:27:52.693 DEBUG:teuthology.orchestra.run.smithi066:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:27:52.772 INFO:teuthology.orchestra.run.smithi066.stdout:ttyS1 2024-09-10T18:27:52.796 DEBUG:teuthology.parallel:result is None 2024-09-10T18:27:52.796 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:27:52.803 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:27:52.803 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:27:52.806 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:27:52.828 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:27:52.834 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:27:52.839 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:27:52.840 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:27:52.866 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:27:52.901 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:27:52.907 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:27:52.908 DEBUG:teuthology.orchestra.run.smithi066:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:27:52.934 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:27:52.968 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:27:52.980 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:27:53.002 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:27:53.015 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:27:53.016 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:27:53.026 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:27:53.026 DEBUG:teuthology.orchestra.run.smithi066:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:27:53.029 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:27:53.057 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:27:53.067 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:27:53.067 DEBUG:teuthology.orchestra.run.smithi066:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:27:53.099 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:27:53.116 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:27:53.201 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:27:53.257 DEBUG:teuthology.orchestra.run.smithi066:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:27:53.339 DEBUG:teuthology.orchestra.run.smithi066:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:27:53.383 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:27:53.383 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:27:53.450 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:27:53.494 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:27:53.546 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:27:53.628 DEBUG:teuthology.orchestra.run.smithi117:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:27:53.673 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:27:53.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:27:53.739 DEBUG:teuthology.orchestra.run.smithi066:> sudo service rsyslog restart 2024-09-10T18:27:53.746 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-09-10T18:27:53.781 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:27:53.825 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:27:54.268 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:27:54.275 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:27:54.275 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:27:54.285 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:27:54.293 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:27:54.293 DEBUG:teuthology.orchestra.run.smithi066:> sudo service auditd rotate 2024-09-10T18:27:54.349 INFO:teuthology.orchestra.run.smithi066.stdout:Rotating logs: 2024-09-10T18:27:54.351 DEBUG:teuthology.orchestra.run.smithi117:> sudo service auditd rotate 2024-09-10T18:27:54.401 INFO:teuthology.orchestra.run.smithi117.stdout:Rotating logs: 2024-09-10T18:27:54.404 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:27:54.404 DEBUG:teuthology.orchestra.run.smithi066:> /usr/sbin/getenforce 2024-09-10T18:27:54.429 INFO:teuthology.orchestra.run.smithi066.stdout:Permissive 2024-09-10T18:27:54.429 DEBUG:teuthology.orchestra.run.smithi117:> /usr/sbin/getenforce 2024-09-10T18:27:54.451 INFO:teuthology.orchestra.run.smithi117.stdout:Permissive 2024-09-10T18:27:54.451 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi066.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi117.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:27:54.452 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:27:54.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:54.498 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:27:54.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:27:54.527 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:27:54.527 DEBUG:teuthology.orchestra.run.smithi066:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:27:54.574 DEBUG:teuthology.orchestra.run.smithi117:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:27:54.607 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:27:54.617 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:27:54.628 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:27:54.629 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi066.front.sepia.ceph.com,smithi117.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:31:40.889 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi066.front.sepia.ceph.com'), Remote(name='ubuntu@smithi117.front.sepia.ceph.com')] 2024-09-10T18:31:40.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:31:40.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi066.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:31:40.971 DEBUG:teuthology.orchestra.run.smithi066:> true 2024-09-10T18:31:41.057 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi066.front.sepia.ceph.com' 2024-09-10T18:31:41.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T18:31:41.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:31:41.136 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-09-10T18:31:41.216 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-09-10T18:31:41.216 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:31:41.227 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:31:41.228 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:31:41.229 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:31:41.233 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:31:41.233 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:31:41.264 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:31:41.281 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:31:41.287 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:31:41.305 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:31:41.307 INFO:teuthology.orchestra.run.smithi066.stderr:sudo: ntpd: command not found 2024-09-10T18:31:41.319 INFO:teuthology.orchestra.run.smithi066.stdout:506 Cannot talk to daemon 2024-09-10T18:31:41.326 INFO:teuthology.orchestra.run.smithi117.stderr:sudo: ntpd: command not found 2024-09-10T18:31:41.336 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:31:41.336 INFO:teuthology.orchestra.run.smithi117.stdout:506 Cannot talk to daemon 2024-09-10T18:31:41.353 INFO:teuthology.orchestra.run.smithi066.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:31:41.355 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:31:41.365 INFO:teuthology.orchestra.run.smithi117.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:31:41.415 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.419 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.430 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.438 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:31:41.438 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:31:41.447 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:31:41.448 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T18:31:41.448 DEBUG:teuthology.orchestra.run.smithi066:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:31:41.448 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T18:31:41.448 DEBUG:teuthology.orchestra.run.smithi117:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:31:41.831 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-10T18:31:41.831 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout:Removing: 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout:Removing unused dependencies: 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:31:41.842 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout:Remove 2 Packages 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout:Freed space: 5.5 M 2024-09-10T18:31:41.843 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-10T18:31:41.844 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-10T18:31:41.844 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:41.844 INFO:teuthology.orchestra.run.smithi117.stdout:Removing: 2024-09-10T18:31:41.844 INFO:teuthology.orchestra.run.smithi117.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:Removing unused dependencies: 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:Remove 2 Packages 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:Freed space: 5.5 M 2024-09-10T18:31:41.845 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-10T18:31:41.846 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-10T18:31:41.847 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-10T18:31:41.849 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-10T18:31:41.849 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-10T18:31:41.907 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-10T18:31:41.907 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-10T18:31:41.920 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-10T18:31:41.920 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-10T18:31:42.049 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-10T18:31:42.069 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-10T18:31:42.157 INFO:teuthology.orchestra.run.smithi117.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:31:42.216 INFO:teuthology.orchestra.run.smithi117.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:31:42.249 INFO:teuthology.orchestra.run.smithi066.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:31:42.302 INFO:teuthology.orchestra.run.smithi066.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:31:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:31:42.709 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:31:42.746 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:31:42.746 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:31:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:31:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:42.931 INFO:teuthology.orchestra.run.smithi117.stdout:Removed: 2024-09-10T18:31:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:31:42.931 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:42.932 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout:Removed: 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:42.982 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-10T18:31:43.778 INFO:teuthology.orchestra.run.smithi117.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:30:51 PM UTC. 2024-09-10T18:31:43.791 INFO:teuthology.orchestra.run.smithi066.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 06:30:56 PM UTC. 2024-09-10T18:31:43.913 INFO:teuthology.orchestra.run.smithi117.stdout:Dependencies resolved. 2024-09-10T18:31:43.913 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: Package Architecture Version Repository Size 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout:Installing: 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout:Installing dependencies: 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:31:43.914 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction Summary 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout:================================================================================ 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout:Install 6 Packages 2024-09-10T18:31:43.915 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:43.921 INFO:teuthology.orchestra.run.smithi117.stdout:Total download size: 2.0 M 2024-09-10T18:31:43.921 INFO:teuthology.orchestra.run.smithi117.stdout:Installed size: 9.2 M 2024-09-10T18:31:43.921 INFO:teuthology.orchestra.run.smithi117.stdout:Downloading Packages: 2024-09-10T18:31:43.928 INFO:teuthology.orchestra.run.smithi066.stdout:Dependencies resolved. 2024-09-10T18:31:43.929 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:43.929 INFO:teuthology.orchestra.run.smithi066.stdout: Package Architecture Version Repository Size 2024-09-10T18:31:43.929 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:43.929 INFO:teuthology.orchestra.run.smithi066.stdout:Installing: 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout:Installing dependencies: 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:31:43.930 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:43.931 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction Summary 2024-09-10T18:31:43.931 INFO:teuthology.orchestra.run.smithi066.stdout:================================================================================ 2024-09-10T18:31:43.931 INFO:teuthology.orchestra.run.smithi066.stdout:Install 6 Packages 2024-09-10T18:31:43.931 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:43.950 INFO:teuthology.orchestra.run.smithi066.stdout:Total download size: 2.0 M 2024-09-10T18:31:43.950 INFO:teuthology.orchestra.run.smithi066.stdout:Installed size: 9.2 M 2024-09-10T18:31:43.950 INFO:teuthology.orchestra.run.smithi066.stdout:Downloading Packages: 2024-09-10T18:31:44.161 INFO:teuthology.orchestra.run.smithi117.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 384 kB/s | 44 kB 00:00 2024-09-10T18:31:44.162 INFO:teuthology.orchestra.run.smithi066.stdout:(1/6): nvmetcli-0.7-3.el9.noarch.rpm 508 kB/s | 44 kB 00:00 2024-09-10T18:31:44.186 INFO:teuthology.orchestra.run.smithi117.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 752 kB/s | 106 kB 00:00 2024-09-10T18:31:44.204 INFO:teuthology.orchestra.run.smithi066.stdout:(2/6): libnvme-1.9-3.el9.x86_64.rpm 819 kB/s | 106 kB 00:00 2024-09-10T18:31:44.236 INFO:teuthology.orchestra.run.smithi117.stdout:(3/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-10T18:31:44.237 INFO:teuthology.orchestra.run.smithi066.stdout:(3/6): python3-configshell-1.1.30-1.el9.noarch. 956 kB/s | 72 kB 00:00 2024-09-10T18:31:44.262 INFO:teuthology.orchestra.run.smithi117.stdout:(4/6): python3-configshell-1.1.30-1.el9.noarch. 714 kB/s | 72 kB 00:00 2024-09-10T18:31:44.262 INFO:teuthology.orchestra.run.smithi066.stdout:(4/6): python3-kmod-0.9-32.el9.x86_64.rpm 1.4 MB/s | 84 kB 00:00 2024-09-10T18:31:44.287 INFO:teuthology.orchestra.run.smithi066.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 4.1 MB/s | 906 kB 00:00 2024-09-10T18:31:44.338 INFO:teuthology.orchestra.run.smithi066.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 8.1 MB/s | 837 kB 00:00 2024-09-10T18:31:44.338 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:44.338 INFO:teuthology.orchestra.run.smithi066.stdout:Total 5.2 MB/s | 2.0 MB 00:00 2024-09-10T18:31:44.403 INFO:teuthology.orchestra.run.smithi117.stdout:(5/6): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.5 MB/s | 906 kB 00:00 2024-09-10T18:31:44.414 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction check 2024-09-10T18:31:44.429 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction check succeeded. 2024-09-10T18:31:44.429 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction test 2024-09-10T18:31:44.487 INFO:teuthology.orchestra.run.smithi117.stdout:(6/6): python3-urwid-2.1.2-4.el9.x86_64.rpm 3.3 MB/s | 837 kB 00:00 2024-09-10T18:31:44.487 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:31:44.487 INFO:teuthology.orchestra.run.smithi117.stdout:Total 3.5 MB/s | 2.0 MB 00:00 2024-09-10T18:31:44.560 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction check 2024-09-10T18:31:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction check succeeded. 2024-09-10T18:31:44.574 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction test 2024-09-10T18:31:44.701 INFO:teuthology.orchestra.run.smithi066.stdout:Transaction test succeeded. 2024-09-10T18:31:44.702 INFO:teuthology.orchestra.run.smithi066.stdout:Running transaction 2024-09-10T18:31:44.865 INFO:teuthology.orchestra.run.smithi117.stdout:Transaction test succeeded. 2024-09-10T18:31:44.865 INFO:teuthology.orchestra.run.smithi117.stdout:Running transaction 2024-09-10T18:31:45.143 INFO:teuthology.orchestra.run.smithi066.stdout: Preparing : 1/1 2024-09-10T18:31:45.225 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:31:45.259 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:31:45.291 INFO:teuthology.orchestra.run.smithi117.stdout: Preparing : 1/1 2024-09-10T18:31:45.296 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:31:45.381 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/6 2024-09-10T18:31:45.416 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/6 2024-09-10T18:31:45.445 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/6 2024-09-10T18:31:45.517 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:31:45.552 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:31:45.651 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/6 2024-09-10T18:31:45.693 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:31:46.053 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:31:46.087 INFO:teuthology.orchestra.run.smithi066.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:31:46.135 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/6 2024-09-10T18:31:46.162 INFO:teuthology.orchestra.run.smithi117.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:31:47.068 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/6 2024-09-10T18:31:47.121 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/6 2024-09-10T18:31:47.416 INFO:teuthology.orchestra.run.smithi066.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout:Installed: 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:31:47.417 INFO:teuthology.orchestra.run.smithi066.stdout:Complete! 2024-09-10T18:31:47.501 INFO:teuthology.orchestra.run.smithi117.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/6 2024-09-10T18:31:47.501 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:47.501 INFO:teuthology.orchestra.run.smithi117.stdout:Installed: 2024-09-10T18:31:47.502 INFO:teuthology.orchestra.run.smithi117.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:31:47.502 INFO:teuthology.orchestra.run.smithi117.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:31:47.502 INFO:teuthology.orchestra.run.smithi117.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:31:47.502 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:31:47.502 INFO:teuthology.orchestra.run.smithi117.stdout:Complete! 2024-09-10T18:31:47.709 DEBUG:teuthology.parallel:result is None 2024-09-10T18:31:47.831 DEBUG:teuthology.parallel:result is None 2024-09-10T18:31:47.831 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-09-10T18:31:47.843 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-09-10T18:31:47.843 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:31:47.843 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:31:47.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:31:47.859 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_1 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:31:46.423156594 +0000 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:46.167137616 +0000 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:46.167137616 +0000 2024-09-10T18:31:47.915 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:46.167137616 +0000 2024-09-10T18:31:47.916 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:31:47.982 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:31:47.982 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:31:47.982 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000212819 s, 2.4 MB/s 2024-09-10T18:31:47.983 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:31:48.041 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_2 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 967 Links: 1 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:31:46.423156594 +0000 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.097 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.097 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:31:48.179 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:31:48.180 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:31:48.180 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000184034 s, 2.8 MB/s 2024-09-10T18:31:48.181 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:31:48.237 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_3 2024-09-10T18:31:48.292 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:31:48.292 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:31:46.424156734 +0000 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.293 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:46.166137625 +0000 2024-09-10T18:31:48.293 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:31:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:31:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:31:48.357 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000167602 s, 3.1 MB/s 2024-09-10T18:31:48.358 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:31:48.415 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/vg_nvme/lv_4 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:31:46.424156734 +0000 2024-09-10T18:31:48.471 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:46.168137607 +0000 2024-09-10T18:31:48.472 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:46.168137607 +0000 2024-09-10T18:31:48.472 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:46.168137607 +0000 2024-09-10T18:31:48.472 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:31:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:31:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:31:48.535 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000192512 s, 2.7 MB/s 2024-09-10T18:31:48.536 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:31:48.594 DEBUG:teuthology.orchestra.run.smithi066:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-10T18:31:48.736 INFO:teuthology.orchestra.run.smithi066.stdout:loop 2024-09-10T18:31:48.738 INFO:tasks.nvme_loop:Connecting nvme_loop smithi066:/dev/vg_nvme/lv_1... 2024-09-10T18:31:48.738 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-10T18:31:48.772 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:48.793 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/vg_nvme/lv_1 2024-09-10T18:31:48.803 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:48.854 INFO:teuthology.orchestra.run.smithi066.stdout:connecting to device: nvme1 2024-09-10T18:31:48.856 INFO:tasks.nvme_loop:Connecting nvme_loop smithi066:/dev/vg_nvme/lv_2... 2024-09-10T18:31:48.856 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-10T18:31:48.933 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:48.954 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/vg_nvme/lv_2 2024-09-10T18:31:48.965 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:49.002 INFO:teuthology.orchestra.run.smithi066.stdout:connecting to device: nvme2 2024-09-10T18:31:49.004 INFO:tasks.nvme_loop:Connecting nvme_loop smithi066:/dev/vg_nvme/lv_3... 2024-09-10T18:31:49.004 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-10T18:31:49.079 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:49.101 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/vg_nvme/lv_3 2024-09-10T18:31:49.112 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:49.149 INFO:teuthology.orchestra.run.smithi066.stdout:connecting to device: nvme3 2024-09-10T18:31:49.151 INFO:tasks.nvme_loop:Connecting nvme_loop smithi066:/dev/vg_nvme/lv_4... 2024-09-10T18:31:49.151 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-10T18:31:49.228 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:49.250 INFO:teuthology.orchestra.run.smithi066.stdout:/dev/vg_nvme/lv_4 2024-09-10T18:31:49.260 INFO:teuthology.orchestra.run.smithi066.stdout:1 2024-09-10T18:31:49.297 INFO:teuthology.orchestra.run.smithi066.stdout:connecting to device: nvme4 2024-09-10T18:31:49.299 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:31:49.299 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:31:49.357 DEBUG:teuthology.orchestra.run.smithi066:> sudo nvme list -o json 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "Devices":[ 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "NameSpace":1, 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "GenericPath":"/dev/ng4n1", 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:49.424 INFO:teuthology.orchestra.run.smithi066.stdout: "SerialNumber":"fe93375e814f916c1593", 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "SectorSize":512 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "NameSpace":1, 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "GenericPath":"/dev/ng3n1", 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:49.425 INFO:teuthology.orchestra.run.smithi066.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "SerialNumber":"5553d225c4aa9749e4e2", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "SectorSize":512 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "NameSpace":1, 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "GenericPath":"/dev/ng2n1", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:49.426 INFO:teuthology.orchestra.run.smithi066.stdout: "SerialNumber":"75aed262aa944d09824a", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "SectorSize":512 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "NameSpace":1, 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "GenericPath":"/dev/ng1n1", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "SerialNumber":"3533e2d361595f882993", 2024-09-10T18:31:49.427 INFO:teuthology.orchestra.run.smithi066.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "SectorSize":512 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: }, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: { 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "NameSpace":1, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "GenericPath":"/dev/ng0n1", 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "Firmware":"8DV101H0", 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "SerialNumber":"CVFT53430091400BGN", 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "UsedBytes":400088457216, 2024-09-10T18:31:49.428 INFO:teuthology.orchestra.run.smithi066.stdout: "MaximumLBA":781422768, 2024-09-10T18:31:49.429 INFO:teuthology.orchestra.run.smithi066.stdout: "PhysicalSize":400088457216, 2024-09-10T18:31:49.429 INFO:teuthology.orchestra.run.smithi066.stdout: "SectorSize":512 2024-09-10T18:31:49.429 INFO:teuthology.orchestra.run.smithi066.stdout: } 2024-09-10T18:31:49.429 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T18:31:49.429 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T18:31:49.430 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T18:31:49.430 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:31:49.430 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/scratch_devs 2024-09-10T18:31:49.496 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:31:49.496 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:31:49.516 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:31:49.517 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-09-10T18:31:49.574 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:31:49.574 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 969 Links: 1 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:31:46.478282852 +0000 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.575 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.575 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:31:49.644 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:31:49.644 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:31:49.644 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000224128 s, 2.3 MB/s 2024-09-10T18:31:49.645 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:31:49.704 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-09-10T18:31:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:31:49.761 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 972 Links: 1 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:31:46.479282787 +0000 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.762 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.762 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:31:49.830 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:31:49.830 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:31:49.830 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000181661 s, 2.8 MB/s 2024-09-10T18:31:49.832 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:31:49.888 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-09-10T18:31:49.944 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:31:46.479282787 +0000 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.945 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:49.945 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:31:50.009 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:31:50.009 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:31:50.009 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00021293 s, 2.4 MB/s 2024-09-10T18:31:50.010 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:31:50.067 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-09-10T18:31:50.123 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:31:50.123 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:31:50.123 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:31:46.480282721 +0000 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:50.124 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:46.242295146 +0000 2024-09-10T18:31:50.124 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:31:50.187 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:31:50.187 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:31:50.187 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000204734 s, 2.5 MB/s 2024-09-10T18:31:50.188 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:31:50.245 DEBUG:teuthology.orchestra.run.smithi117:> sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-09-10T18:31:50.384 INFO:teuthology.orchestra.run.smithi117.stdout:loop 2024-09-10T18:31:50.385 INFO:tasks.nvme_loop:Connecting nvme_loop smithi117:/dev/vg_nvme/lv_1... 2024-09-10T18:31:50.385 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-09-10T18:31:50.417 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.437 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_1 2024-09-10T18:31:50.447 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.486 INFO:teuthology.orchestra.run.smithi117.stdout:connecting to device: nvme1 2024-09-10T18:31:50.492 INFO:tasks.nvme_loop:Connecting nvme_loop smithi117:/dev/vg_nvme/lv_2... 2024-09-10T18:31:50.492 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-09-10T18:31:50.567 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.589 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_2 2024-09-10T18:31:50.599 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.631 INFO:teuthology.orchestra.run.smithi117.stdout:connecting to device: nvme2 2024-09-10T18:31:50.633 INFO:tasks.nvme_loop:Connecting nvme_loop smithi117:/dev/vg_nvme/lv_3... 2024-09-10T18:31:50.633 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-09-10T18:31:50.707 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.727 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_3 2024-09-10T18:31:50.737 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.768 INFO:teuthology.orchestra.run.smithi117.stdout:connecting to device: nvme3 2024-09-10T18:31:50.770 INFO:tasks.nvme_loop:Connecting nvme_loop smithi117:/dev/vg_nvme/lv_4... 2024-09-10T18:31:50.770 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-09-10T18:31:50.844 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.864 INFO:teuthology.orchestra.run.smithi117.stdout:/dev/vg_nvme/lv_4 2024-09-10T18:31:50.874 INFO:teuthology.orchestra.run.smithi117.stdout:1 2024-09-10T18:31:50.905 INFO:teuthology.orchestra.run.smithi117.stdout:connecting to device: nvme4 2024-09-10T18:31:50.906 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:31:50.906 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:31:50.963 DEBUG:teuthology.orchestra.run.smithi117:> sudo nvme list -o json 2024-09-10T18:31:51.029 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T18:31:51.029 INFO:teuthology.orchestra.run.smithi117.stdout: "Devices":[ 2024-09-10T18:31:51.029 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "NameSpace":1, 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "DevicePath":"/dev/nvme4n1", 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "GenericPath":"/dev/ng4n1", 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "SerialNumber":"2efa744e6b687b594f4e", 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: "SectorSize":512 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T18:31:51.030 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "NameSpace":1, 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "DevicePath":"/dev/nvme3n1", 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "GenericPath":"/dev/ng3n1", 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "SerialNumber":"74bf8d7c2a1fd726ed33", 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: "SectorSize":512 2024-09-10T18:31:51.031 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "NameSpace":1, 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "DevicePath":"/dev/nvme2n1", 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "GenericPath":"/dev/ng2n1", 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "SerialNumber":"4d6ffbfc4584fd33863c", 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:51.032 INFO:teuthology.orchestra.run.smithi117.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "SectorSize":512 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "NameSpace":1, 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "DevicePath":"/dev/nvme1n1", 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "GenericPath":"/dev/ng1n1", 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "Firmware":"5.14.0-5", 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "ModelNumber":"Linux", 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "SerialNumber":"f97121855214c7129da4", 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "UsedBytes":95995035648, 2024-09-10T18:31:51.033 INFO:teuthology.orchestra.run.smithi117.stdout: "MaximumLBA":187490304, 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "PhysicalSize":95995035648, 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "SectorSize":512 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: }, 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: { 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "NameSpace":1, 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "DevicePath":"/dev/nvme0n1", 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "GenericPath":"/dev/ng0n1", 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "Firmware":"8DV101H0", 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "ModelNumber":"INTEL SSDPEDMD400G4", 2024-09-10T18:31:51.034 INFO:teuthology.orchestra.run.smithi117.stdout: "SerialNumber":"PHFT620400V6400BGN", 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: "UsedBytes":400088457216, 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: "MaximumLBA":781422768, 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: "PhysicalSize":400088457216, 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: "SectorSize":512 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: } 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-10T18:31:51.035 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T18:31:51.036 INFO:tasks.nvme_loop:new_devs ['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T18:31:51.036 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:31:51.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/scratch_devs 2024-09-10T18:31:51.101 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:31:51.201 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': True}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'MON_DOWN', 'mons down', 'mon down', 'out of quorum'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:31:51.201 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:31:51.202 INFO:tasks.cephadm:Cluster fsid is f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:31:51.202 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:31:51.202 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.66', 'mon.c': '[v2:172.21.15.66:3301,v1:172.21.15.66:6790]', 'mon.b': '172.21.15.117'} 2024-09-10T18:31:51.202 INFO:tasks.cephadm:First mon is mon.a on smithi066 2024-09-10T18:31:51.202 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:31:51.203 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:31:51.203 DEBUG:teuthology.orchestra.run.smithi066:> sudo hostname $(hostname -s) 2024-09-10T18:31:51.235 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-09-10T18:31:51.260 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:31:51.260 DEBUG:teuthology.orchestra.run.smithi066:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:31:51.294 INFO:teuthology.orchestra.run.smithi066.stderr:Cloning into 'testrepo'... 2024-09-10T18:32:50.087 INFO:teuthology.orchestra.run.smithi066.stderr:Updating files: 91% (11591/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T18:32:50.122 INFO:teuthology.orchestra.run.smithi066.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:32 /home/ubuntu/cephtest/cephadm 2024-09-10T18:32:50.123 DEBUG:teuthology.orchestra.run.smithi117:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:32:50.143 INFO:teuthology.orchestra.run.smithi117.stderr:Cloning into 'testrepo'... 2024-09-10T18:33:48.388 INFO:teuthology.orchestra.run.smithi117.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:33 /home/ubuntu/cephtest/cephadm 2024-09-10T18:33:48.389 DEBUG:teuthology.orchestra.run.smithi066:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:33:48.410 DEBUG:teuthology.orchestra.run.smithi117:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:33:48.455 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:33:48.455 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:33:48.458 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:33:49.981 INFO:teuthology.orchestra.run.smithi066.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:33:51.823 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:34:17.331 INFO:teuthology.orchestra.run.smithi066.stdout:{ 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "repo_digests": [ 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout: ] 2024-09-10T18:34:17.332 INFO:teuthology.orchestra.run.smithi066.stdout:} 2024-09-10T18:34:18.231 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-09-10T18:34:18.232 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-09-10T18:34:18.249 DEBUG:teuthology.orchestra.run.smithi066:> sudo mkdir -p /etc/ceph 2024-09-10T18:34:18.282 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-09-10T18:34:18.318 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 777 /etc/ceph 2024-09-10T18:34:18.350 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-09-10T18:34:18.383 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:34:18.384 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:34:18.384 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = True 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:34:18.385 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:34:18.386 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:34:18.386 DEBUG:teuthology.orchestra.run.smithi066:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:34:18.408 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f248e234-6fa2-11ef-bcea-c7b262605968 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = True [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-10T18:34:18.408 DEBUG:teuthology.orchestra.run.smithi066:mon.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:34:18.451 DEBUG:teuthology.orchestra.run.smithi066:mgr.y> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:34:18.494 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:34:18.494 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid f248e234-6fa2-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.66 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:18.706 INFO:teuthology.orchestra.run.smithi066.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:34:18.706 INFO:teuthology.orchestra.run.smithi066.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'f248e234-6fa2-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.66', '--skip-admin-label'] 2024-09-10T18:34:18.731 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:34:18.731 INFO:teuthology.orchestra.run.smithi066.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:34:18.731 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying podman|docker is present... 2024-09-10T18:34:18.761 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:34:18.761 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying lvm2 is present... 2024-09-10T18:34:18.761 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying time synchronization is in place... 2024-09-10T18:34:18.769 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:34:18.770 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:34:18.776 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:34:18.777 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:34:18.784 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2024-09-10T18:34:18.791 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2024-09-10T18:34:18.791 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:34:18.792 INFO:teuthology.orchestra.run.smithi066.stdout:Repeating the final host check... 2024-09-10T18:34:18.815 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:34:18.816 INFO:teuthology.orchestra.run.smithi066.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:34:18.816 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl is present 2024-09-10T18:34:18.816 INFO:teuthology.orchestra.run.smithi066.stdout:lvcreate is present 2024-09-10T18:34:18.823 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:34:18.823 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:34:18.830 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:34:18.830 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:34:18.838 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout enabled 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout active 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:Host looks OK 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:Cluster fsid: f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:Acquiring lock 140152305533328 on /run/cephadm/f248e234-6fa2-11ef-bcea-c7b262605968.lock 2024-09-10T18:34:18.846 INFO:teuthology.orchestra.run.smithi066.stdout:Lock 140152305533328 acquired on /run/cephadm/f248e234-6fa2-11ef-bcea-c7b262605968.lock 2024-09-10T18:34:18.847 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 3300 ... 2024-09-10T18:34:18.847 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying IP 172.21.15.66 port 6789 ... 2024-09-10T18:34:18.847 INFO:teuthology.orchestra.run.smithi066.stdout:Base mon IP(s) is [172.21.15.66:3300, 172.21.15.66:6789], mon addrv is [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2024-09-10T18:34:18.852 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.66 metric 100 2024-09-10T18:34:18.853 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.66 metric 100 2024-09-10T18:34:18.854 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:34:18.854 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T18:34:18.854 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:feb6:f8bd/64 scope link noprefixroute 2024-09-10T18:34:18.857 INFO:teuthology.orchestra.run.smithi066.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:34:18.858 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2024-09-10T18:34:18.858 INFO:teuthology.orchestra.run.smithi066.stdout:Mon IP `172.21.15.66` is in CIDR network `172.21.0.0/20` 2024-09-10T18:34:18.862 INFO:teuthology.orchestra.run.smithi066.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T18:34:18.862 INFO:teuthology.orchestra.run.smithi066.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:34:18.862 INFO:teuthology.orchestra.run.smithi066.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:34:19.354 INFO:teuthology.orchestra.run.smithi066.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:34:20.643 INFO:teuthology.orchestra.run.smithi066.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:34:20.643 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:34:20.644 INFO:teuthology.orchestra.run.smithi066.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:34:21.631 INFO:teuthology.orchestra.run.smithi066.stdout:stat: stdout 167 167 2024-09-10T18:34:21.631 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial keys... 2024-09-10T18:34:22.520 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQAtkeBmJ+/TORAAOEpSfmRoTW8uFmiQZSkpfw== 2024-09-10T18:34:23.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQAukeBmR56xMhAAnaZ/6/5e6vhlYN16rKWZcg== 2024-09-10T18:34:24.347 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph-authtool: stdout AQAvkeBm/z/5LBAAT6AWtun+Qm0++AmY57dZqw== 2024-09-10T18:34:24.348 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial monmap... 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:monmaptool for a [v2:172.21.15.66:3300,v1:172.21.15.66:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:34:25.253 INFO:teuthology.orchestra.run.smithi066.stdout:setting min_mon_release = octopus 2024-09-10T18:34:25.254 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: set fsid to f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:25.254 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:34:25.254 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:34:25.254 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mon... 2024-09-10T18:34:26.292 INFO:teuthology.orchestra.run.smithi066.stdout:create mon.a on 2024-09-10T18:34:26.701 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:34:26.865 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target. 2024-09-10T18:34:26.865 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target. 2024-09-10T18:34:27.052 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:34:27.052 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:34:27.230 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target.wants/ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968@.service. 2024-09-10T18:34:27.782 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:34:27.782 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-09-10T18:34:27.789 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:34:27.789 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:34:27.789 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-09-10T18:34:27.789 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:34:27.789 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon to start... 2024-09-10T18:34:27.790 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mon... 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout id: f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.55915s) 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:28.761 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:mon is available 2024-09-10T18:34:28.762 INFO:teuthology.orchestra.run.smithi066.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:34:29.816 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:29.816 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.66:3300,v1:172.21.15.66:6789] 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = True 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:34:29.817 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:29.818 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:34:29.818 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:34:29.818 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:34:29.818 INFO:teuthology.orchestra.run.smithi066.stdout:Generating new minimal ceph.conf... 2024-09-10T18:34:30.805 INFO:teuthology.orchestra.run.smithi066.stdout:Restarting the monitor... 2024-09-10T18:34:30.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:30 smithi066 systemd[1]: Stopping Ceph mon.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:34:31.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a[25031]: 2024-09-10T18:34:31.078+0000 7f951488f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:34:31.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a[25031]: 2024-09-10T18:34:31.078+0000 7f951488f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:34:31.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 podman[25277]: 2024-09-10 18:34:31.140441291 +0000 UTC m=+0.171786288 container died e64d508d3b6c5344e7f6ee8d25d8ac8039ed4b8dae9b759fd5d7cf15a7679f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T18:34:31.655 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 podman[25277]: 2024-09-10 18:34:31.469823756 +0000 UTC m=+0.501168753 container cleanup e64d508d3b6c5344e7f6ee8d25d8ac8039ed4b8dae9b759fd5d7cf15a7679f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T18:34:31.655 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 bash[25277]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:34:31.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 podman[25290]: 2024-09-10 18:34:31.654114018 +0000 UTC m=+0.508010876 container remove e64d508d3b6c5344e7f6ee8d25d8ac8039ed4b8dae9b759fd5d7cf15a7679f58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:34:31.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:34:31.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 systemd[1]: Stopped Ceph mon.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:34:31.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:31 smithi066 systemd[1]: Starting Ceph mon.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:34:32.375 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 podman[25380]: 2024-09-10 18:34:32.004841151 +0000 UTC m=+0.015837310 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 podman[25380]: 2024-09-10 18:34:32.113498757 +0000 UTC m=+0.124494904 container create a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 podman[25380]: 2024-09-10 18:34:32.298863141 +0000 UTC m=+0.309859286 container init a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 podman[25380]: 2024-09-10 18:34:32.302518995 +0000 UTC m=+0.313515139 container start a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: pidfile_write: ignore empty --pid-file 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: load: jerasure load: lrc 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: DB SUMMARY 2024-09-10T18:34:32.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: DB Session ID: 9KK3CTJYFLE01CWNQA2C 2024-09-10T18:34:32.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:34:32.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:34:32.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:34:32.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:34:32.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 82860 ; 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.env: 0x55d4f5a4d440 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.fs: Posix File System 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.info_log: 0x55d4f645eec0 2024-09-10T18:34:32.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.statistics: (nil) 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.use_fsync: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:34:32.379 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.db_log_dir: 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.write_buffer_manager: 0x55d4f651c7e0 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:34:32.380 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.unordered_write: 0 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:34:32.381 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.row_cache: None 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.wal_filter: None 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:34:32.382 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:34:32.383 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_open_files: -1 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Compression algorithms supported: 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kZSTD supported: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:34:32.384 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.merge_operator: 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_filter: None 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:34:32.385 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d4f6405a38) 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_type: 0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_index_type: 0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_shortening: 1 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: hash_index_allow_collision: 1 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: checksum: 1 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: no_block_cache: 0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache: 0x55d4f643b1f0 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:34:32.386 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_options: 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: capacity : 536870912 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: num_shard_bits : 4 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: strict_capacity_limit : 0 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_cache_compressed: (nil) 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: persistent_cache: (nil) 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size: 4096 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_size_deviation: 10 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_restart_interval: 16 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: index_block_restart_interval: 1 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: metadata_block_size: 4096 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: partition_filters: 0 2024-09-10T18:34:32.387 INFO:journalctl@ceph.mon.a.smithi066.stdout: use_delta_encoding: 1 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: whole_key_filtering: 1 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: verify_compression: 0 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: format_version: 4 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_index_compression: 1 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout: block_align: 0 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression: NoCompression 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:34:32.388 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.num_levels: 7 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:34:32.389 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:34:32.390 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:34:32.391 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:34:32.392 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:34:32.393 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.ttl: 2592000 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:34:32.394 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993272325963, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993272326755, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 79590, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 77822, "index_size": 230, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 9759, "raw_average_key_size": 47, "raw_value_size": 72229, "raw_average_value_size": 348, "num_data_blocks": 10, "num_entries": 207, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993272, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "20d0a15c-a238-4cf3-9d39-3716ba497d20", "db_session_id": "9KK3CTJYFLE01CWNQA2C"}} 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993272327141, "job": 1, "event": "recovery_finished"} 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55d4f646e700 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: DB pointer 0x55d4f643e000 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:34:32.395 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** DB Stats ** 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: L0 2/0 79.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.396 INFO:journalctl@ceph.mon.a.smithi066.stdout: Sum 2/0 79.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative compaction: 0.00 GB write, 23.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:32.397 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval compaction: 0.00 GB write, 23.46 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: L0 2/0 79.51 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: Sum 2/0 79.51 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:34:32.398 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.1 0.00 0.00 1 0.001 0 0 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: Cumulative compaction: 0.00 GB write, 23.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:32.399 INFO:journalctl@ceph.mon.a.smithi066.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: starting mon.a rank 0 at public addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] at bind addrs [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???) e1 preinit fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).mds e1 new map 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: e1 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: legacy client fscid: -1 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout: No filesystems configured 2024-09-10T18:34:32.400 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 bash[25380]: a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 2024-09-10T18:34:32.401 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 systemd[1]: Started Ceph mon.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:34:32.401 INFO:teuthology.orchestra.run.smithi066.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:34:32.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:34:32.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: monmap e1: 1 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:34:32.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: fsmap 2024-09-10T18:34:32.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:34:32.626 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:32 smithi066 ceph-mon[25398]: mgrmap e1: no daemons active 2024-09-10T18:34:33.401 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:34:33.401 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:34:33.401 INFO:teuthology.orchestra.run.smithi066.stdout:Creating mgr... 2024-09-10T18:34:33.401 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:34:33.401 INFO:teuthology.orchestra.run.smithi066.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:34:33.589 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:34:33.589 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:34:33.748 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968.target.wants/ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-f248e234-6fa2-11ef-bcea-c7b262605968@.service. 2024-09-10T18:34:33.853 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:33 smithi066 systemd[1]: Starting Ceph mgr.y for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:34:34.124 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:33 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2152887668' entity='client.admin' 2024-09-10T18:34:34.124 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:33 smithi066 podman[25612]: 2024-09-10 18:34:33.888405373 +0000 UTC m=+0.016239453 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:34:34.125 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:33 smithi066 podman[25612]: 2024-09-10 18:34:33.989352587 +0000 UTC m=+0.117186652 container create 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T18:34:34.341 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:34:34.341 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-09-10T18:34:34.349 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:34:34.349 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:34:34.349 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-09-10T18:34:34.349 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:34:34.358 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:34:34.358 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr to start... 2024-09-10T18:34:34.366 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr... 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 podman[25612]: 2024-09-10 18:34:34.237109471 +0000 UTC m=+0.364943537 container init 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 podman[25612]: 2024-09-10 18:34:34.239964982 +0000 UTC m=+0.367799048 container start 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 bash[25612]: 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 systemd[1]: Started Ceph mgr.y for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:34.359+0000 7f4b1867eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:34.420+0000 7f4b1867eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:34:34.490 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:34.487+0000 7f4b1867eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:34:34.765 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:34.619+0000 7f4b1867eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:34:34.765 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:34.762+0000 7f4b1867eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:34:35.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:34 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2847444131' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:34:35.432 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:35.040+0000 7f4b1867eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:34:35.432 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:35.103+0000 7f4b1867eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:34:35.433 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:35.272+0000 7f4b1867eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "f248e234-6fa2-11ef-bcea-c7b262605968", 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:34:35.440 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:34:35.441 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:34:35.442 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.443 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:35.444 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:34:27.706652+0000", 2024-09-10T18:34:35.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:35.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:35.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:34:35.445 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:35.445 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:34:36.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:36.153+0000 7f4b1867eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:34:36.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:36.220+0000 7f4b1867eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:34:37.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:36.965+0000 7f4b1867eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:34:37.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.085+0000 7f4b1867eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:34:37.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.212+0000 7f4b1867eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:34:37.540 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.274+0000 7f4b1867eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:34:37.541 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.405+0000 7f4b1867eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:34:37.541 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.478+0000 7f4b1867eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:34:37.541 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.538+0000 7f4b1867eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:34:37.801 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.681+0000 7f4b1867eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:34:37.801 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.748+0000 7f4b1867eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:34:38.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/371289788' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:34:38.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.817+0000 7f4b1867eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:34:38.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.903+0000 7f4b1867eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:34:38.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:37.965+0000 7f4b1867eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:34:38.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:38.026+0000 7f4b1867eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:34:38.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:38.521 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "f248e234-6fa2-11ef-bcea-c7b262605968", 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:38.522 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:34:38.523 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:34:38.524 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:34:38.525 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:34:27.706652+0000", 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:38.526 INFO:teuthology.orchestra.run.smithi066.stdout:mgr not available, waiting (2/15)... 2024-09-10T18:34:39.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:38.811+0000 7f4b1867eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:34:39.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: Activating manager daemon y 2024-09-10T18:34:39.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: mgrmap e2: y(active, starting, since 0.00280565s) 2024-09-10T18:34:39.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: Manager daemon y is now available 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' 2024-09-10T18:34:39.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:38 smithi066 ceph-mon[25398]: from='mgr.14100 172.21.15.66:0/537024605' entity='mgr.y' 2024-09-10T18:34:40.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:40 smithi066 ceph-mon[25398]: mgrmap e3: y(active, since 1.00735s) 2024-09-10T18:34:41.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:41 smithi066 ceph-mon[25398]: mgrmap e4: y(active, since 2s) 2024-09-10T18:34:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:43 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1366864486' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:34:49.389 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsid": "f248e234-6fa2-11ef-bcea-c7b262605968", 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:34:49.390 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.391 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:34:49.392 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:34:49.395 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.395 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:34:49.395 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:34:49.395 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:34:49.395 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:49.396 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.397 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:34:27.706652+0000", 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:49.398 INFO:teuthology.orchestra.run.smithi066.stdout:mgr is available 2024-09-10T18:34:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:50 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3070166430' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:34:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:50 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3070166430' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:34:50.509 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:50.509 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:34:50.509 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout fsid = f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:34:50.509 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:34:50.510 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling cephadm module... 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/462526116' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setuser ceph since I am not root 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setgroup ceph since I am not root 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.289+0000 7f8b54102ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.346+0000 7f8b54102ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:34:51.418 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.415+0000 7f8b54102ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:34:51.687 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.544+0000 7f8b54102ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:34:51.964 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.685+0000 7f8b54102ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:34:51.964 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:51 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:51.962+0000 7f8b54102ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:34:52.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:52 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/462526116' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:34:52.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:52 smithi066 ceph-mon[25398]: mgrmap e5: y(active, since 12s) 2024-09-10T18:34:52.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:52 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:52.027+0000 7f8b54102ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:34:52.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:52 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:52.198+0000 7f8b54102ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:34:52.995 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2024-09-10T18:34:52.996 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 5... 2024-09-10T18:34:53.271 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:53 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:53.087+0000 7f8b54102ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:34:53.271 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:53 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:53.156+0000 7f8b54102ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:34:53.271 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:53 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1437277023' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:34:54.201 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:53 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:53.896+0000 7f8b54102ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:34:54.201 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.012+0000 7f8b54102ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:34:54.201 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.137+0000 7f8b54102ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:34:54.201 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.198+0000 7f8b54102ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:34:54.461 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.329+0000 7f8b54102ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:34:54.461 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.398+0000 7f8b54102ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:34:54.461 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.458+0000 7f8b54102ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:34:54.728 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.598+0000 7f8b54102ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:34:54.728 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.665+0000 7f8b54102ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:34:55.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.733+0000 7f8b54102ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:34:55.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.819+0000 7f8b54102ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:34:55.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.879+0000 7f8b54102ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:34:55.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:54 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:54.944+0000 7f8b54102ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:55.722+0000 7f8b54102ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: Active manager daemon y restarted 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: Activating manager daemon y 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: mgrmap e6: y(active, starting, since 0.00369269s) 2024-09-10T18:34:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: Manager daemon y is now available 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:34:55.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:55 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:34:57.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:56 smithi066 ceph-mon[25398]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:34:57.028 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:56 smithi066 ceph-mon[25398]: mgrmap e7: y(active, since 1.00553s) 2024-09-10T18:34:57.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:34:57.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:34:57.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:34:57.313 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:34:57.313 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 5 is available 2024-09-10T18:34:57.314 INFO:teuthology.orchestra.run.smithi066.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:34:57.905 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:57 smithi066 ceph-mon[25398]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:34:57.905 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:57 smithi066 ceph-mon[25398]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:34:58.995 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:58 smithi066 ceph-mon[25398]: mgrmap e8: y(active, since 2s) 2024-09-10T18:34:58.996 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:58 smithi066 ceph-mon[25398]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:58.996 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:58 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:34:58.996 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:58 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:34:59.483 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:34:59.483 INFO:teuthology.orchestra.run.smithi066.stdout:Generating ssh key... 2024-09-10T18:34:59.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:59.254+0000 7f8b064b0640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:34:59] ENGINE Error in HTTPServer.serve 2024-09-10T18:34:59.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Traceback (most recent call last): 2024-09-10T18:34:59.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:34:59.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._connections.run(self.expiration_interval) 2024-09-10T18:34:59.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._run(expiration_interval) 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s = self.context.wrap_socket( 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: return self.sslsocket_class._create( 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self.do_handshake() 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:34:59.605 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._sslobj.do_handshake() 2024-09-10T18:34:59.606 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:34:59.606 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:34:59.900 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-mon[25398]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:34:59.900 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-mon[25398]: [10/Sep/2024:18:34:59] ENGINE Bus STARTING 2024-09-10T18:34:59.900 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:34:59.900 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:34:59 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Generating public/private rsa key pair. 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Your identification has been saved in /tmp/tmpvidbta9i/key 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Your public key has been saved in /tmp/tmpvidbta9i/key.pub 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: The key fingerprint is: 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: SHA256:Kjck2v09Sr4T9SZasSr0u99GWJUXr4YxbfySEsUN1Qg ceph-f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: The key's randomart image is: 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: +---[RSA 3072]----+ 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | EooB+| 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | += =| 2024-09-10T18:35:00.554 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | +.+..| 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | o .* + | 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | . . S. *o = .| 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | o +... = +o . | 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | . o.=..= + | 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | o.==.... | 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: | .BBoo. | 2024-09-10T18:35:00.555 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: +----[SHA256]-----+ 2024-09-10T18:35:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: [10/Sep/2024:18:34:59] ENGINE Serving on https://172.21.15.66:7150 2024-09-10T18:35:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: [10/Sep/2024:18:34:59] ENGINE Bus STARTED 2024-09-10T18:35:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: [10/Sep/2024:18:34:59] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout: Traceback (most recent call last): 2024-09-10T18:35:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._run(expiration_interval) 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: s = self.context.wrap_socket( 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: return self.sslsocket_class._create( 2024-09-10T18:35:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout: self.do_handshake() 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._sslobj.do_handshake() 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: Generating ssh key... 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:00 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:02.237 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ40MiHsHr331nCEWtUzzx+98ErrywTdNX6bQIAh0DiuvxBui912cgiZCk88Ys8HsZBYAHgSLpo/Lar45zKbLpammqGM1i5YEmTGIEfyl1o+HIAhx+vclaVZbMAq2COIs9Vp0yf7WnRNj9t0hQKOaLbumFw0veqVeQtxnBbdxK1vLLK3SCCk3GNAWZxXmdS3k7B2VuR/2j5crcpnw6GenlpyPMu+ICCeMWkaMyiW52opV471xbB4TDE9NuPeowzPjkYqTJtT4NYJMSls8uoe9oCUJFoPo95RUbYAu4qpzgq5YeLaYCeW702vyFn9U/aLdsh7yIwa3LFf3kp+LRdvWvWW/cocbvpdH/tIP+c3kxNR8sMJhsu9wC06YmKwl91ZaKYMZzohk02kmtPFbdEeive92j05guo+1TVAku/MAgZOCvugFNzuRA/CJ32XK6ywqYlKCF0Y6e4IELGoLglT8hSBGR2GZa6PsyYp/7bW9rIRNpMgo0x3yrK3/hXxJgBn0= ceph-f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:02.237 INFO:teuthology.orchestra.run.smithi066.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:35:02.238 INFO:teuthology.orchestra.run.smithi066.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:35:02.242 INFO:teuthology.orchestra.run.smithi066.stdout:Adding host smithi066... 2024-09-10T18:35:02.470 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:02 smithi066 ceph-mon[25398]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:03.525 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:03 smithi066 ceph-mon[25398]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi066", "addr": "172.21.15.66", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:04.425 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:04 smithi066 ceph-mon[25398]: Deploying cephadm binary to smithi066 2024-09-10T18:35:06.846 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Added host 'smithi066' with addr '172.21.15.66' 2024-09-10T18:35:06.846 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mon service... 2024-09-10T18:35:07.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:07 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:07.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:07 smithi066 ceph-mon[25398]: Added host smithi066 2024-09-10T18:35:07.448 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:07 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:08.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:08 smithi066 ceph-mon[25398]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:08.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:08 smithi066 ceph-mon[25398]: Saving service mon spec with placement count:5 2024-09-10T18:35:08.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:08 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:08.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:08 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:08.869 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:35:08.869 INFO:teuthology.orchestra.run.smithi066.stdout:Deploying unmanaged mgr service... 2024-09-10T18:35:10.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:09 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:10.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:09 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:10.285 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:35:11.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:10 smithi066 ceph-mon[25398]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:11.107 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:10 smithi066 ceph-mon[25398]: Saving service mgr spec with placement count:2 2024-09-10T18:35:12.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:12.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:12.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:12.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' 2024-09-10T18:35:12.043 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi066", "caps": []}]: dispatch 2024-09-10T18:35:12.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='mgr.14118 172.21.15.66:0/3691581742' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi066", "caps": []}]': finished 2024-09-10T18:35:12.044 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:11 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1748406254' entity='client.admin' 2024-09-10T18:35:12.799 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling the dashboard module... 2024-09-10T18:35:13.215 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:12 smithi066 ceph-mon[25398]: Deploying daemon agent.smithi066 on smithi066 2024-09-10T18:35:13.216 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:12 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/923908848' entity='client.admin' 2024-09-10T18:35:14.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:13 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2461451691' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:35:14.229 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setuser ceph since I am not root 2024-09-10T18:35:14.229 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setgroup ceph since I am not root 2024-09-10T18:35:14.229 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.119+0000 7f35af3caec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:35:14.229 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.176+0000 7f35af3caec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:14.502 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.244+0000 7f35af3caec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:35:14.502 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.373+0000 7f35af3caec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:14.794 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.516+0000 7f35af3caec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:35:14.794 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.791+0000 7f35af3caec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:35:15.134 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2461451691' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:35:15.135 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-mon[25398]: mgrmap e9: y(active, since 18s) 2024-09-10T18:35:15.135 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:14.855+0000 7f35af3caec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:35:15.135 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:15 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:15.027+0000 7f35af3caec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for the mgr to restart... 2024-09-10T18:35:15.729 INFO:teuthology.orchestra.run.smithi066.stdout:Waiting for mgr epoch 9... 2024-09-10T18:35:16.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:15 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/921725375' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:35:16.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:15 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:15.928+0000 7f35af3caec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:35:16.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:16 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:15.998+0000 7f35af3caec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:35:17.042 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:16 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:16.738+0000 7f35af3caec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:35:17.043 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:16 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:16.854+0000 7f35af3caec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:35:17.043 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:16 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:16.979+0000 7f35af3caec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:35:17.043 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.040+0000 7f35af3caec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:35:17.302 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.170+0000 7f35af3caec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:35:17.302 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.240+0000 7f35af3caec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:35:17.302 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.300+0000 7f35af3caec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:35:17.574 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.439+0000 7f35af3caec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:35:17.575 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.505+0000 7f35af3caec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:35:17.575 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.572+0000 7f35af3caec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:35:17.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.656+0000 7f35af3caec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:35:17.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.715+0000 7f35af3caec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:35:17.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:17 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:17.774+0000 7f35af3caec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:18.550+0000 7f35af3caec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: Active manager daemon y restarted 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: Activating manager daemon y 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: mgrmap e10: y(active, starting, since 0.00304528s) 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:35:18.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: Manager daemon y is now available 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:35:18.806 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:35:20.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:20.399+0000 7f355ff27640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:35:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Traceback (most recent call last): 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._connections.run(self.expiration_interval) 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._run(expiration_interval) 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:20.479 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s = self.context.wrap_socket( 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: return self.sslsocket_class._create( 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self.do_handshake() 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._sslobj.do_handshake() 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:20.480 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:35:20.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:35:20.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:35:20.505 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:35:20.506 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:35:20.506 INFO:teuthology.orchestra.run.smithi066.stdout:mgr epoch 9 is available 2024-09-10T18:35:20.506 INFO:teuthology.orchestra.run.smithi066.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:35:20.804 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: mgrmap e11: y(active, since 1.0057s) 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi066", "caps": []}]: dispatch 2024-09-10T18:35:20.805 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:21.753 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:35:21.753 INFO:teuthology.orchestra.run.smithi066.stdout:Creating initial admin user... 2024-09-10T18:35:21.874 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: [10/Sep/2024:18:35:20] ENGINE Bus STARTING 2024-09-10T18:35:21.874 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: [10/Sep/2024:18:35:20] ENGINE Serving on https://172.21.15.66:7150 2024-09-10T18:35:21.874 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: [10/Sep/2024:18:35:20] ENGINE Bus STARTED 2024-09-10T18:35:21.874 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: [10/Sep/2024:18:35:20] ENGINE Error in HTTPServer.serve 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: Traceback (most recent call last): 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._run(expiration_interval) 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: s = self.context.wrap_socket( 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:35:21.875 INFO:journalctl@ceph.mon.a.smithi066.stdout: return self.sslsocket_class._create( 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout: self.do_handshake() 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._sslobj.do_handshake() 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: Deploying daemon agent.smithi066 on smithi066 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:21.876 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:21 smithi066 ceph-mon[25398]: mgrmap e12: y(active, since 2s) 2024-09-10T18:35:22.932 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:22 smithi066 ceph-mon[25398]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:23.209 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$IfOIzmf4JPIBbqO7b71iXehRdLE7lKTew/fGtQCI/ThK5yIhHULra", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725993322, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:35:23.209 INFO:teuthology.orchestra.run.smithi066.stdout:Fetching dashboard port number... 2024-09-10T18:35:23.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:23 smithi066 ceph-mon[25398]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:23.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:24.423 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:35:24.434 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:35:24.434 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout disabled 2024-09-10T18:35:24.442 INFO:teuthology.orchestra.run.smithi066.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:35:24.442 INFO:teuthology.orchestra.run.smithi066.stdout:systemctl: stdout inactive 2024-09-10T18:35:24.442 INFO:teuthology.orchestra.run.smithi066.stdout:firewalld.service is not enabled 2024-09-10T18:35:24.442 INFO:teuthology.orchestra.run.smithi066.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout:Ceph Dashboard is now available at: 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout: URL: https://smithi066.front.sepia.ceph.com:8443/ 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout: User: admin 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout: Password: 552wy1jjek 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:24.445 INFO:teuthology.orchestra.run.smithi066.stdout:Saving cluster configuration to /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config directory 2024-09-10T18:35:24.446 INFO:teuthology.orchestra.run.smithi066.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:35:24.679 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:24 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3251198653' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:35:25.896 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:25 smithi066 ceph-mon[25398]: mgrmap e13: y(active, since 6s) 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:35:26.795 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: ceph telemetry on 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout:For more information see: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:26.796 INFO:teuthology.orchestra.run.smithi066.stdout:Bootstrap complete. 2024-09-10T18:35:26.992 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:35:26.992 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:35:26.992 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:35:27.008 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:35:27.008 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:35:27.008 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:35:27.069 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:35:27.069 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:35:27.069 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:35:27.191 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:35:27.191 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:35:27.191 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:35:27.242 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:27 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1888184774' entity='client.admin' 2024-09-10T18:35:27.242 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:27.242 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:27.250 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:35:27.250 DEBUG:teuthology.orchestra.run.smithi066:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ40MiHsHr331nCEWtUzzx+98ErrywTdNX6bQIAh0DiuvxBui912cgiZCk88Ys8HsZBYAHgSLpo/Lar45zKbLpammqGM1i5YEmTGIEfyl1o+HIAhx+vclaVZbMAq2COIs9Vp0yf7WnRNj9t0hQKOaLbumFw0veqVeQtxnBbdxK1vLLK3SCCk3GNAWZxXmdS3k7B2VuR/2j5crcpnw6GenlpyPMu+ICCeMWkaMyiW52opV471xbB4TDE9NuPeowzPjkYqTJtT4NYJMSls8uoe9oCUJFoPo95RUbYAu4qpzgq5YeLaYCeW702vyFn9U/aLdsh7yIwa3LFf3kp+LRdvWvWW/cocbvpdH/tIP+c3kxNR8sMJhsu9wC06YmKwl91ZaKYMZzohk02kmtPFbdEeive92j05guo+1TVAku/MAgZOCvugFNzuRA/CJ32XK6ywqYlKCF0Y6e4IELGoLglT8hSBGR2GZa6PsyYp/7bW9rIRNpMgo0x3yrK3/hXxJgBn0= ceph-f248e234-6fa2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:35:27.505 INFO:teuthology.orchestra.run.smithi066.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ40MiHsHr331nCEWtUzzx+98ErrywTdNX6bQIAh0DiuvxBui912cgiZCk88Ys8HsZBYAHgSLpo/Lar45zKbLpammqGM1i5YEmTGIEfyl1o+HIAhx+vclaVZbMAq2COIs9Vp0yf7WnRNj9t0hQKOaLbumFw0veqVeQtxnBbdxK1vLLK3SCCk3GNAWZxXmdS3k7B2VuR/2j5crcpnw6GenlpyPMu+ICCeMWkaMyiW52opV471xbB4TDE9NuPeowzPjkYqTJtT4NYJMSls8uoe9oCUJFoPo95RUbYAu4qpzgq5YeLaYCeW702vyFn9U/aLdsh7yIwa3LFf3kp+LRdvWvWW/cocbvpdH/tIP+c3kxNR8sMJhsu9wC06YmKwl91ZaKYMZzohk02kmtPFbdEeive92j05guo+1TVAku/MAgZOCvugFNzuRA/CJ32XK6ywqYlKCF0Y6e4IELGoLglT8hSBGR2GZa6PsyYp/7bW9rIRNpMgo0x3yrK3/hXxJgBn0= ceph-f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:27.700 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ40MiHsHr331nCEWtUzzx+98ErrywTdNX6bQIAh0DiuvxBui912cgiZCk88Ys8HsZBYAHgSLpo/Lar45zKbLpammqGM1i5YEmTGIEfyl1o+HIAhx+vclaVZbMAq2COIs9Vp0yf7WnRNj9t0hQKOaLbumFw0veqVeQtxnBbdxK1vLLK3SCCk3GNAWZxXmdS3k7B2VuR/2j5crcpnw6GenlpyPMu+ICCeMWkaMyiW52opV471xbB4TDE9NuPeowzPjkYqTJtT4NYJMSls8uoe9oCUJFoPo95RUbYAu4qpzgq5YeLaYCeW702vyFn9U/aLdsh7yIwa3LFf3kp+LRdvWvWW/cocbvpdH/tIP+c3kxNR8sMJhsu9wC06YmKwl91ZaKYMZzohk02kmtPFbdEeive92j05guo+1TVAku/MAgZOCvugFNzuRA/CJ32XK6ywqYlKCF0Y6e4IELGoLglT8hSBGR2GZa6PsyYp/7bW9rIRNpMgo0x3yrK3/hXxJgBn0= ceph-f248e234-6fa2-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:35:27.757 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCZ40MiHsHr331nCEWtUzzx+98ErrywTdNX6bQIAh0DiuvxBui912cgiZCk88Ys8HsZBYAHgSLpo/Lar45zKbLpammqGM1i5YEmTGIEfyl1o+HIAhx+vclaVZbMAq2COIs9Vp0yf7WnRNj9t0hQKOaLbumFw0veqVeQtxnBbdxK1vLLK3SCCk3GNAWZxXmdS3k7B2VuR/2j5crcpnw6GenlpyPMu+ICCeMWkaMyiW52opV471xbB4TDE9NuPeowzPjkYqTJtT4NYJMSls8uoe9oCUJFoPo95RUbYAu4qpzgq5YeLaYCeW702vyFn9U/aLdsh7yIwa3LFf3kp+LRdvWvWW/cocbvpdH/tIP+c3kxNR8sMJhsu9wC06YmKwl91ZaKYMZzohk02kmtPFbdEeive92j05guo+1TVAku/MAgZOCvugFNzuRA/CJ32XK6ywqYlKCF0Y6e4IELGoLglT8hSBGR2GZa6PsyYp/7bW9rIRNpMgo0x3yrK3/hXxJgBn0= ceph-f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:27.770 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:35:28.059 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:28.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:28.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:28.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:29.840 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:31.525 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:35:31.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:35:31.790 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:31 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2883586417' entity='client.admin' 2024-09-10T18:35:31.790 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:31.790 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:31.790 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:31.790 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:31.990 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:34.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:34.949 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi117 2024-09-10T18:35:34.949 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:35:34.949 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:35:34.968 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:35:34.968 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:35.027 INFO:tasks.cephadm:Adding host smithi117 to orchestrator... 2024-09-10T18:35:35.027 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch host add smithi117 2024-09-10T18:35:35.277 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:35.680 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:35.681 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:37.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:37 smithi066 ceph-mon[25398]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:39.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:39 smithi066 ceph-mon[25398]: Deploying cephadm binary to smithi117 2024-09-10T18:35:39.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:39 smithi066 ceph-mon[25398]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:40.280 INFO:teuthology.orchestra.run.smithi066.stdout:Added host 'smithi117' with addr '172.21.15.117' 2024-09-10T18:35:40.864 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:35:41.111 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:41.559 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: Added host smithi117 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:41.560 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:41 smithi066 ceph-mon[25398]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:35:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]: dispatch 2024-09-10T18:35:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]': finished 2024-09-10T18:35:42.848 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:35:42.848 INFO:teuthology.orchestra.run.smithi066.stdout:[{"addr": "172.21.15.66", "hostname": "smithi066", "labels": [], "status": ""}, {"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}] 2024-09-10T18:35:43.452 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:35:43.453 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:35:43.697 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:35:44.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: Deploying daemon agent.smithi117 on smithi117 2024-09-10T18:35:44.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.115 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.116 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:44.116 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:45.925 INFO:teuthology.orchestra.run.smithi066.stderr:adjusted tunables profile to default 2024-09-10T18:35:46.223 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:45 smithi066 ceph-mon[25398]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:46.223 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:45 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:35:46.223 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:46.223 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:46.516 INFO:tasks.cephadm:Adding mon.a on smithi066 2024-09-10T18:35:46.517 INFO:tasks.cephadm:Adding mon.c on smithi066 2024-09-10T18:35:46.517 INFO:tasks.cephadm:Adding mon.b on smithi117 2024-09-10T18:35:46.517 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b' 2024-09-10T18:35:46.750 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:47.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:46 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:35:47.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:46 smithi066 ceph-mon[25398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:35:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:47 smithi066 ceph-mon[25398]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:48.377 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mon update... 2024-09-10T18:35:48.931 DEBUG:teuthology.orchestra.run.smithi066:mon.c> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:35:48.934 DEBUG:teuthology.orchestra.run.smithi117:mon.b> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:35:48.936 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:35:48.936 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b;count:3 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:49.484 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: Deploying daemon mon.b on smithi117 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:49.485 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:49 smithi066 ceph-mon[25398]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:49.638 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:51.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:51 smithi066 ceph-mon[25398]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:51.899 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:35:51.899 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":1,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","modified":"2024-09-10T18:34:24.674156Z","created":"2024-09-10T18:34:24.674156Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:35:51.900 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 1 2024-09-10T18:35:52.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:52 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/901849918' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:35:53.689 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:35:53.689 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:35:53.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:53 smithi066 ceph-mon[25398]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:54.081 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:35:55.624 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 podman[27412]: 2024-09-10 18:35:55.603865015 +0000 UTC m=+0.572106885 container start 103585c9d90a40a1be44d7c9105351c179ec558958a324f7455855e38a7d5ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:35:55.624 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:35:55.624 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:35:55.624 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pidfile_write: ignore empty --pid-file 2024-09-10T18:35:55.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:35:55 smithi066 ceph-mon[25398]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: load: jerasure load: lrc 2024-09-10T18:35:56.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:35:56.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:35:56.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:35:56.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: DB SUMMARY 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: DB Session ID: ZQVCOCDV68KVLKGPWWWF 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.env: 0x56107360d440 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.fs: Posix File System 2024-09-10T18:35:56.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.info_log: 0x56107509cec0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.statistics: (nil) 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.use_fsync: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:35:56.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.db_log_dir: 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:35:56.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.write_buffer_manager: 0x56107515a7e0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:35:56.071 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.unordered_write: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.row_cache: None 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.wal_filter: None 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:35:56.072 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:35:56.073 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_open_files: -1 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:35:56.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Compression algorithms supported: 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kZSTD supported: 0 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:35:56.075 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.merge_operator: 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_filter: None 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561075043a38) 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:35:56.076 INFO:journalctl@ceph.mon.b.smithi117.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_type: 0 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_index_type: 0 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_shortening: 1 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: hash_index_allow_collision: 1 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: checksum: 1 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: no_block_cache: 0 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache: 0x561075079350 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_options: 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: capacity : 536870912 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: num_shard_bits : 4 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: strict_capacity_limit : 0 2024-09-10T18:35:56.077 INFO:journalctl@ceph.mon.b.smithi117.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_cache_compressed: (nil) 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: persistent_cache: (nil) 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size: 4096 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_size_deviation: 10 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_restart_interval: 16 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: index_block_restart_interval: 1 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: metadata_block_size: 4096 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: partition_filters: 0 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: use_delta_encoding: 1 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: whole_key_filtering: 1 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: verify_compression: 0 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:35:56.078 INFO:journalctl@ceph.mon.b.smithi117.stdout: format_version: 4 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_index_compression: 1 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout: block_align: 0 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression: NoCompression 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.num_levels: 7 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:35:56.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:35:56.080 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:35:56.081 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:35:56.082 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:35:56.083 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.ttl: 2592000 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:35:56.084 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993355628077, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993355628376, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993355, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "cf28ba16-2ff3-48ad-8a80-0b7908edd44a", "db_session_id": "ZQVCOCDV68KVLKGPWWWF"}} 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993355628735, "job": 1, "event": "recovery_finished"} 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5610750ac700 2024-09-10T18:35:56.085 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: DB pointer 0x56107507c000 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** DB Stats ** 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T18:35:56.086 INFO:journalctl@ceph.mon.b.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:35:56.087 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval compaction: 0.00 GB write, 0.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.088 INFO:journalctl@ceph.mon.b.smithi117.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** Compaction Stats [default] ** 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.3 0.00 0.00 1 0.000 0 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:35:56.089 INFO:journalctl@ceph.mon.b.smithi117.stdout: Cumulative compaction: 0.00 GB write, 0.55 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: using public_addr v2:172.21.15.117:0/0 -> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: starting mon.b rank -1 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(???) e0 preinit fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: e1 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:35:56.090 INFO:journalctl@ceph.mon.b.smithi117.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout: legacy client fscid: -1 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout: 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout: No filesystems configured 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:35:56.091 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]: dispatch 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]': finished 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Deploying daemon agent.smithi117 on smithi117 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.092 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.093 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b;count:3 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:35:56.094 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: Deploying daemon mon.b on smithi117 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/901849918' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:35:56.096 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 ceph-mon[27457]: mon.b@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-10T18:35:56.096 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 bash[27412]: 103585c9d90a40a1be44d7c9105351c179ec558958a324f7455855e38a7d5ae9 2024-09-10T18:35:56.096 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:35:55 smithi117 systemd[1]: Started Ceph mon.b for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:36:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: mon.a calling monitor election 2024-09-10T18:36:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: mon.b calling monitor election 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: fsmap 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:00.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: mgrmap e13: y(active, since 42s) 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: overall HEALTH_OK 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:00.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: mon.a calling monitor election 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: mon.b calling monitor election 2024-09-10T18:36:01.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: fsmap 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: mgrmap e13: y(active, since 42s) 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: overall HEALTH_OK 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:01.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:01.068 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:01.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:01 smithi117 ceph-mon[27457]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:01.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:01 smithi117 ceph-mon[27457]: Deploying daemon mon.c on smithi066 2024-09-10T18:36:01.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:01 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:01.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:01 smithi066 ceph-mon[25398]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:01.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:01 smithi066 ceph-mon[25398]: Deploying daemon mon.c on smithi066 2024-09-10T18:36:01.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:01 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:02.107 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:36:02.107 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":2,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","modified":"2024-09-10T18:35:55.658388Z","created":"2024-09-10T18:34:24.674156Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:36:02.107 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 2 2024-09-10T18:36:03.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:02 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/4248586866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:03.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:02 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/4248586866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:03.662 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:36:03.663 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:36:03.678 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:03 smithi066 systemd[1]: Starting Ceph mon.c for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:36:03.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:03 smithi066 ceph-mon[25398]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:03.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:03 smithi066 podman[31670]: 2024-09-10 18:36:03.643924947 +0000 UTC m=+0.016810407 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:36:03.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:03 smithi066 podman[31670]: 2024-09-10 18:36:03.781802099 +0000 UTC m=+0.154687549 container create 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T18:36:04.057 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:03 smithi117 ceph-mon[27457]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.371 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:36:04.456 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 podman[31670]: 2024-09-10 18:36:04.0006738 +0000 UTC m=+0.373559251 container init 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 podman[31670]: 2024-09-10 18:36:04.004008564 +0000 UTC m=+0.376894017 container start 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 2 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pidfile_write: ignore empty --pid-file 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: load: jerasure load: lrc 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: DB SUMMARY 2024-09-10T18:36:04.457 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: DB Session ID: FGI18OERHRDRQR9NUNV0 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.env: 0x561430ecd440 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.fs: Posix File System 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.info_log: 0x56143200cec0 2024-09-10T18:36:04.458 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:36:04.459 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.statistics: (nil) 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.use_fsync: 0 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:36:04.460 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.db_log_dir: 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:36:04.461 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.write_buffer_manager: 0x5614320ca7b0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:36:04.462 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.unordered_write: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.row_cache: None 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.wal_filter: None 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:36:04.463 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:36:04.464 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_open_files: -1 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:36:04.465 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Compression algorithms supported: 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kZSTD supported: 0 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:36:04.466 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.merge_operator: 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_filter: None 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561431fb3a38) 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: index_type: 0 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: data_block_index_type: 0 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: index_shortening: 1 2024-09-10T18:36:04.467 INFO:journalctl@ceph.mon.c.smithi066.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: hash_index_allow_collision: 1 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: checksum: 1 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: no_block_cache: 0 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_cache: 0x561431fe91f0 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_cache_options: 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: capacity : 536870912 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: num_shard_bits : 4 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: strict_capacity_limit : 0 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_cache_compressed: (nil) 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: persistent_cache: (nil) 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_size: 4096 2024-09-10T18:36:04.468 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_size_deviation: 10 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_restart_interval: 16 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: index_block_restart_interval: 1 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: metadata_block_size: 4096 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: partition_filters: 0 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: use_delta_encoding: 1 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: whole_key_filtering: 1 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: verify_compression: 0 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: format_version: 4 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: enable_index_compression: 1 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout: block_align: 0 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:36:04.469 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression: NoCompression 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.num_levels: 7 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:36:04.470 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:36:04.471 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:36:04.472 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:36:04.473 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:36:04.474 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.ttl: 2592000 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:36:04.475 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993364028185, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993364028514, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725993364, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f4f9f194-ad52-4690-ae81-de59fe353502", "db_session_id": "FGI18OERHRDRQR9NUNV0"}} 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725993364028851, "job": 1, "event": "recovery_finished"} 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x56143201c700 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: DB pointer 0x561431fec000 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** DB Stats ** 2024-09-10T18:36:04.476 INFO:journalctl@ceph.mon.c.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.477 INFO:journalctl@ceph.mon.c.smithi066.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:36:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:36:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** Compaction Stats [default] ** 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.2 0.00 0.00 1 0.000 0 0 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:36:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: using public_addrv [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: starting mon.c rank -1 at public addrs [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] at bind addrs [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(???) e0 preinit fsid f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: e1 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:36:04.481 INFO:journalctl@ceph.mon.c.smithi066.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout: legacy client fscid: -1 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout: 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout: No filesystems configured 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:36:04.482 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:04.483 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]: dispatch 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.agent.smithi117", "caps": []}]': finished 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Deploying daemon agent.smithi117 on smithi117 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.484 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.485 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2456169802' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Saving service mon spec with placement smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b;count:3 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.486 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Deploying daemon mon.b on smithi117 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/901849918' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.a calling monitor election 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.487 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.b calling monitor election 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: monmap e2: 2 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: fsmap 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mgrmap e13: y(active, since 42s) 2024-09-10T18:36:04.488 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: overall HEALTH_OK 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: Deploying daemon mon.c on smithi066 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/4248586866' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 ceph-mon[31695]: mon.c@-1(synchronizing).paxosservice(auth 1..5) refresh upgraded, format 0 -> 3 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 bash[31670]: 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa 2024-09-10T18:36:04.489 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:04 smithi066 systemd[1]: Started Ceph mon.c for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: mon.a calling monitor election 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: mon.b calling monitor election 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.371 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: fsmap 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: mgrmap e13: y(active, since 50s) 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: mon.c (rank 2) addr [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] is down (out of quorum) 2024-09-10T18:36:09.372 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.373 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.468 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-09-10T18:36:09.468 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":3,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","modified":"2024-09-10T18:36:04.051582Z","created":"2024-09-10T18:34:24.674156Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3300","nonce":0},{"type":"v1","addr":"172.21.15.66:6789","nonce":0}]},"addr":"172.21.15.66:6789/0","public_addr":"172.21.15.66:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:3301","nonce":0},{"type":"v1","addr":"172.21.15.66:6790","nonce":0}]},"addr":"172.21.15.66:6790/0","public_addr":"172.21.15.66:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:36:09.469 INFO:teuthology.orchestra.run.smithi117.stderr:dumped monmap epoch 3 2024-09-10T18:36:09.470 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:09.470 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: mon.a calling monitor election 2024-09-10T18:36:09.470 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: mon.b calling monitor election 2024-09-10T18:36:09.470 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:09.470 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.471 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:09.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:09.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:09.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:09.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: fsmap 2024-09-10T18:36:09.472 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: mgrmap e13: y(active, since 50s) 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: mon.c (rank 2) addr [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] is down (out of quorum) 2024-09-10T18:36:09.473 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.474 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.475 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.475 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:09.475 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.056 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:36:10.057 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:36:10.376 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:10.377 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/106083535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: Reconfiguring daemon mon.a on smithi066 2024-09-10T18:36:10.378 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:10.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/106083535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: Reconfiguring daemon mon.a on smithi066 2024-09-10T18:36:10.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.317 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.a calling monitor election 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.b calling monitor election 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.318 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: fsmap 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mgrmap e13: y(active, since 50s) 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.c (rank 2) addr [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] is down (out of quorum) 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.319 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/106083535' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:11.321 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:11.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:11.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Reconfiguring daemon mon.a on smithi066 2024-09-10T18:36:11.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mon.c calling monitor election 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mon.c calling monitor election 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mon.b calling monitor election 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mon.a calling monitor election 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: fsmap 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: mgrmap e13: y(active, since 52s) 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:36:11.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: Cluster is now healthy 2024-09-10T18:36:11.807 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:11 smithi117 ceph-mon[27457]: overall HEALTH_OK 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mon.c calling monitor election 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mon.c calling monitor election 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mon.b calling monitor election 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mon.a calling monitor election 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:11.930 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: fsmap 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: mgrmap e13: y(active, since 52s) 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: Cluster is now healthy 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[25398]: overall HEALTH_OK 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.c calling monitor election 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.c calling monitor election 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.b calling monitor election 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.a calling monitor election 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:36:11.931 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: monmap e3: 3 mons at {a=[v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0],b=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],c=[v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: fsmap 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: mgrmap e13: y(active, since 52s) 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: Cluster is now healthy 2024-09-10T18:36:11.932 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:11 smithi066 ceph-mon[31695]: overall HEALTH_OK 2024-09-10T18:36:12.916 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:12.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.957 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:12.958 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:12.958 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:12.958 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:13.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.112 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:13.113 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:13.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:36:13.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: Reconfiguring daemon mon.b on smithi117 2024-09-10T18:36:13.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:13.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: Reconfiguring daemon mon.b on smithi117 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:13.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:13 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: Reconfiguring daemon mon.b on smithi117 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:14.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:13 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: Reconfiguring daemon mon.c on smithi066 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:15.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: Reconfiguring daemon mon.c on smithi066 2024-09-10T18:36:15.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:15.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:15.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: Reconfiguring daemon mon.c on smithi066 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:15.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:16.720 INFO:teuthology.orchestra.run.smithi066.stdout:# minimal ceph.conf for f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:36:16.721 INFO:teuthology.orchestra.run.smithi066.stdout:[global] 2024-09-10T18:36:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: fsid = f248e234-6fa2-11ef-bcea-c7b262605968 2024-09-10T18:36:16.721 INFO:teuthology.orchestra.run.smithi066.stdout: mon_host = [v2:172.21.15.66:3300/0,v1:172.21.15.66:6789/0] [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.66:3301/0,v1:172.21.15.66:6790/0] 2024-09-10T18:36:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:17 smithi066 ceph-mon[25398]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:17 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2173001994' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:17.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:17 smithi066 ceph-mon[31695]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:17.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:17 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2173001994' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:18.039 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:36:18.039 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:36:18.039 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:36:18.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:17 smithi117 ceph-mon[27457]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:18.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:17 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2173001994' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:18.068 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:36:18.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:36:18.134 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:36:18.135 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:36:18.168 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:36:18.168 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:36:18.238 INFO:tasks.cephadm:Adding mgr.y on smithi066 2024-09-10T18:36:18.238 INFO:tasks.cephadm:Adding mgr.x on smithi117 2024-09-10T18:36:18.238 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi066=y;smithi117=x' 2024-09-10T18:36:18.517 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:36:18.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:18 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:18.954 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:18 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:18.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:18.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:18.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:18.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:19.809 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:19 smithi117 ceph-mon[27457]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:19.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:19 smithi066 ceph-mon[31695]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:19.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:19 smithi066 ceph-mon[25398]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:20.182 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled mgr update... 2024-09-10T18:36:20.864 DEBUG:teuthology.orchestra.run.smithi117:mgr.x> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:36:20.866 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:36:20.866 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:36:20.866 DEBUG:teuthology.orchestra.run.smithi066:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:36:20.884 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T18:36:20.884 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/nvme4n1 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/nvme4n1 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 1039 Links: 1 Device type: 103,8 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:36:18.763875942 +0000 2024-09-10T18:36:20.941 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:49.301259507 +0000 2024-09-10T18:36:20.942 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:49.301259507 +0000 2024-09-10T18:36:20.942 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:49.296259551 +0000 2024-09-10T18:36:20.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-10T18:36:21.007 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:36:21.007 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:36:21.007 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000228172 s, 2.2 MB/s 2024-09-10T18:36:21.008 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-10T18:36:21.065 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/nvme3n1 2024-09-10T18:36:21.121 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/nvme3n1 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:36:18.753876030 +0000 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:49.153260819 +0000 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:49.153260819 +0000 2024-09-10T18:36:21.122 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:49.148260864 +0000 2024-09-10T18:36:21.122 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-10T18:36:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:36:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:36:21.189 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.000212771 s, 2.4 MB/s 2024-09-10T18:36:21.191 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-10T18:36:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi066=y;smithi117=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: Saving service mgr spec with placement smithi066=y;smithi117=x;count:2 2024-09-10T18:36:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: Deploying daemon mgr.x on smithi117 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[25398]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:21.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi066=y;smithi117=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: Saving service mgr spec with placement smithi066=y;smithi117=x;count:2 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: Deploying daemon mgr.x on smithi117 2024-09-10T18:36:21.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:21 smithi066 ceph-mon[31695]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:21.246 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/nvme2n1 2024-09-10T18:36:21.302 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/nvme2n1 2024-09-10T18:36:21.302 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:21.302 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 1016 Links: 1 Device type: 103,4 2024-09-10T18:36:21.302 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:21.303 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:21.303 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:36:18.743876118 +0000 2024-09-10T18:36:21.303 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:49.010262088 +0000 2024-09-10T18:36:21.303 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:49.010262088 +0000 2024-09-10T18:36:21.303 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:49.005262132 +0000 2024-09-10T18:36:21.303 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-10T18:36:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:36:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:36:21.366 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00019658 s, 2.6 MB/s 2024-09-10T18:36:21.368 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-10T18:36:21.424 DEBUG:teuthology.orchestra.run.smithi066:> stat /dev/nvme1n1 2024-09-10T18:36:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: File: /dev/nvme1n1 2024-09-10T18:36:21.479 INFO:teuthology.orchestra.run.smithi066.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:21.479 INFO:teuthology.orchestra.run.smithi066.stdout:Device: 5h/5d Inode: 1003 Links: 1 Device type: 103,2 2024-09-10T18:36:21.479 INFO:teuthology.orchestra.run.smithi066.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:21.480 INFO:teuthology.orchestra.run.smithi066.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:21.480 INFO:teuthology.orchestra.run.smithi066.stdout:Access: 2024-09-10 18:36:18.732876215 +0000 2024-09-10T18:36:21.480 INFO:teuthology.orchestra.run.smithi066.stdout:Modify: 2024-09-10 18:31:48.858263436 +0000 2024-09-10T18:36:21.480 INFO:teuthology.orchestra.run.smithi066.stdout:Change: 2024-09-10 18:31:48.858263436 +0000 2024-09-10T18:36:21.480 INFO:teuthology.orchestra.run.smithi066.stdout: Birth: 2024-09-10 18:31:48.853263480 +0000 2024-09-10T18:36:21.480 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-10T18:36:21.543 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records in 2024-09-10T18:36:21.543 INFO:teuthology.orchestra.run.smithi066.stderr:1+0 records out 2024-09-10T18:36:21.543 INFO:teuthology.orchestra.run.smithi066.stderr:512 bytes copied, 0.00022659 s, 2.3 MB/s 2024-09-10T18:36:21.544 DEBUG:teuthology.orchestra.run.smithi066:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-10T18:36:21.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='client.24106 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi066=y;smithi117=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:21.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: Saving service mgr spec with placement smithi066=y;smithi117=x;count:2 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: Deploying daemon mgr.x on smithi117 2024-09-10T18:36:21.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:21 smithi117 ceph-mon[27457]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:21.601 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:36:21.601 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:36:21.617 DEBUG:teuthology.misc:devs=['/dev/nvme4n1', '/dev/nvme3n1', '/dev/nvme2n1', '/dev/nvme1n1'] 2024-09-10T18:36:21.617 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/nvme4n1 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/nvme4n1 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 1040 Links: 1 Device type: 103,8 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:36:12.826822409 +0000 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:50.907154528 +0000 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:50.907154528 +0000 2024-09-10T18:36:21.673 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:50.903154610 +0000 2024-09-10T18:36:21.673 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-09-10T18:36:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:36:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:36:21.738 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000232531 s, 2.2 MB/s 2024-09-10T18:36:21.739 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-09-10T18:36:21.795 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/nvme3n1 2024-09-10T18:36:21.850 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/nvme3n1 2024-09-10T18:36:21.850 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:21.850 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 1027 Links: 1 Device type: 103,6 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:36:12.815822634 +0000 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:50.772157288 +0000 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:50.772157288 +0000 2024-09-10T18:36:21.851 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:50.767157390 +0000 2024-09-10T18:36:21.851 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-09-10T18:36:22.039 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:36:22.039 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:36:22.039 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000202887 s, 2.5 MB/s 2024-09-10T18:36:22.041 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-09-10T18:36:22.057 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/nvme2n1 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/nvme2n1 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 1015 Links: 1 Device type: 103,4 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:36:12.804822858 +0000 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:50.635160088 +0000 2024-09-10T18:36:22.121 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:50.635160088 +0000 2024-09-10T18:36:22.122 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:50.629160211 +0000 2024-09-10T18:36:22.122 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-09-10T18:36:22.231 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:36:22.231 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:36:22.231 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.00019812 s, 2.6 MB/s 2024-09-10T18:36:22.232 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-09-10T18:36:22.247 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/nvme1n1 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/nvme1n1 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 1004 Links: 1 Device type: 103,2 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-09-10T18:36:22.303 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-09-10 18:36:12.794823062 +0000 2024-09-10T18:36:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-09-10 18:31:50.489163072 +0000 2024-09-10T18:36:22.304 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-09-10 18:31:50.489163072 +0000 2024-09-10T18:36:22.304 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: 2024-09-10 18:31:50.485163154 +0000 2024-09-10T18:36:22.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-09-10T18:36:22.369 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-09-10T18:36:22.370 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-09-10T18:36:22.370 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000228918 s, 2.2 MB/s 2024-09-10T18:36:22.371 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-09-10T18:36:22.427 INFO:tasks.cephadm:Deploying osd.0 on smithi066 with /dev/nvme1n1... 2024-09-10T18:36:22.428 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-10T18:36:22.684 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:23.004 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:22.873+0000 7ff32fb0bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:36:23.004 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:23.002+0000 7ff32fb0bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:36:23.315 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:23.136+0000 7ff32fb0bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:36:23.801 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:23.391+0000 7ff32fb0bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:36:23.801 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:23.449+0000 7ff32fb0bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:36:23.801 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:23.608+0000 7ff32fb0bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: Reconfiguring daemon mgr.y on smithi066 2024-09-10T18:36:24.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.139 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:24.139 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.139 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.139 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: Reconfiguring daemon mgr.y on smithi066 2024-09-10T18:36:24.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.141 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: Reconfiguring daemon mgr.y on smithi066 2024-09-10T18:36:24.142 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:24.815 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:24.437+0000 7ff32fb0bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:36:24.815 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:24.500+0000 7ff32fb0bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:36:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/nvme1n1 2024-09-10T18:36:25.180 INFO:teuthology.orchestra.run.smithi066.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:36:25.184 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-10T18:36:25.197 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-09-10T18:36:25.198 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-09-10T18:36:25.198 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115969 s, 904 MB/s 2024-09-10T18:36:25.198 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-09-10T18:36:25.455 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.222+0000 7ff32fb0bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:36:25.455 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.333+0000 7ff32fb0bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:36:25.711 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.453+0000 7ff32fb0bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:36:25.711 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.515+0000 7ff32fb0bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:36:25.711 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.642+0000 7ff32fb0bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:36:25.711 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.709+0000 7ff32fb0bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:36:25.990 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.767+0000 7ff32fb0bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:36:25.990 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.907+0000 7ff32fb0bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:36:25.990 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:25.988+0000 7ff32fb0bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:36:25.990 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:25.991 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:26.066 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:26.067 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:26.068 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:26.068 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi066:/dev/nvme1n1 2024-09-10T18:36:26.256 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:26 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:26.055+0000 7ff32fb0bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:36:26.256 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:26 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:26.135+0000 7ff32fb0bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:36:26.256 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:26 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:26.196+0000 7ff32fb0bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:36:26.256 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:26 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:26.254+0000 7ff32fb0bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:36:26.316 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:26.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:26 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:26.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:26 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:27.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:26 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:36:27.539 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:36:27.092+0000 7ff32fb0bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:36:27.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:27.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: Standby manager daemon x started 2024-09-10T18:36:27.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:36:27.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: Standby manager daemon x started 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:36:27.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:28.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: Standby manager daemon x started 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2683717955' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:36:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: mgrmap e14: y(active, since 69s), standbys: x 2024-09-10T18:36:29.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:36:29.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: mgrmap e14: y(active, since 69s), standbys: x 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:29.231 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.231 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: mgrmap e14: y(active, since 69s), standbys: x 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:29.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:29 smithi066 ceph-mon[31695]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:29.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:29 smithi066 ceph-mon[25398]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:30.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:29 smithi117 ceph-mon[27457]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]: dispatch 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]': finished 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[31695]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]: dispatch 2024-09-10T18:36:31.123 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]': finished 2024-09-10T18:36:31.124 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[25398]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:36:31.124 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:30 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:31.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:30 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]: dispatch 2024-09-10T18:36:31.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:30 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1150658262' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "686cf549-884c-444b-a3c4-d1f69af43911"}]': finished 2024-09-10T18:36:31.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:30 smithi117 ceph-mon[27457]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:36:31.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:30 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:32.085 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:31 smithi066 ceph-mon[25398]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:32.085 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:31 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1287202226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:32.085 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:31 smithi066 ceph-mon[31695]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:32.085 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:31 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1287202226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:32.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:31 smithi117 ceph-mon[27457]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:32.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:31 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1287202226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:33.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:33 smithi066 ceph-mon[25398]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:33.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:33 smithi066 ceph-mon[31695]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:34.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:33 smithi117 ceph-mon[27457]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:35.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[25398]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:35.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:35.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:35.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:35.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[31695]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:35.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:35.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:35.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:35 smithi117 ceph-mon[27457]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:38.022 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[25398]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:38.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:36:38.023 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:38.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[31695]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:38.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:36:38.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:37 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:38.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:37 smithi117 ceph-mon[27457]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:37 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:36:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:37 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:39.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:38 smithi066 ceph-mon[25398]: Deploying daemon osd.0 on smithi066 2024-09-10T18:36:39.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:38 smithi066 ceph-mon[31695]: Deploying daemon osd.0 on smithi066 2024-09-10T18:36:39.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:38 smithi117 ceph-mon[27457]: Deploying daemon osd.0 on smithi066 2024-09-10T18:36:40.168 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:39 smithi066 ceph-mon[25398]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:40.168 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:39 smithi066 ceph-mon[31695]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:40.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:39 smithi117 ceph-mon[27457]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:42.142 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:41 smithi117 ceph-mon[27457]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:42.222 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:41 smithi066 ceph-mon[25398]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:42.223 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:41 smithi066 ceph-mon[31695]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.023 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.024 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:43.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:43.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:43.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:43.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:44.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:43 smithi066 ceph-mon[25398]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:44.222 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:43 smithi066 ceph-mon[31695]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:44.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:43 smithi117 ceph-mon[27457]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:45.974 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:45 smithi117 ceph-mon[27457]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:45.974 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:45 smithi117 ceph-mon[27457]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:36:46.080 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 0 on host 'smithi066' 2024-09-10T18:36:46.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:45 smithi066 ceph-mon[25398]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:46.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:45 smithi066 ceph-mon[25398]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:36:46.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:45 smithi066 ceph-mon[31695]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:46.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:45 smithi066 ceph-mon[31695]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:36:46.674 DEBUG:teuthology.orchestra.run.smithi066:osd.0> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:36:46.676 INFO:tasks.cephadm:Deploying osd.1 on smithi066 with /dev/nvme2n1... 2024-09-10T18:36:46.676 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:46.961 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:46.962 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.962 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.962 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.962 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.966 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.979 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0[36230]: 2024-09-10T18:36:46.848+0000 7f3835582640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:36:46.979 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0[36230]: 2024-09-10T18:36:46.853+0000 7f38303b9640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:36:46.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:46.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi066 to 15040M 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:47 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi066 to 15040M 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi066 to 15040M 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: from='osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:47 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:48.764 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/nvme2n1 2024-09-10T18:36:48.764 INFO:teuthology.orchestra.run.smithi066.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:36:48.768 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-10T18:36:48.784 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-09-10T18:36:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-09-10T18:36:48.784 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131378 s, 798 MB/s 2024-09-10T18:36:48.784 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832] boot 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:36:49.105 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832] boot 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:36:49.106 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:36:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:36:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: osd.0 [v2:172.21.15.66:6802/4045628832,v1:172.21.15.66:6803/4045628832] boot 2024-09-10T18:36:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:36:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:36:49.419 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi066:/dev/nvme2n1 2024-09-10T18:36:49.674 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:36:50.109 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:49 smithi066 ceph-mon[25398]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:50.109 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:49 smithi066 ceph-mon[25398]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:36:50.109 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:49 smithi066 ceph-mon[31695]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:50.109 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:49 smithi066 ceph-mon[31695]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:36:50.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:49 smithi117 ceph-mon[27457]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:36:50.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:49 smithi117 ceph-mon[27457]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:36:52.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[31695]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:52.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:52.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:52.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:52.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[25398]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:52.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:52.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:52.114 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:51 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:52.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:51 smithi117 ceph-mon[27457]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:51 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:36:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:51 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:36:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:51 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:36:53.178 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:52 smithi066 ceph-mon[25398]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:53.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:52 smithi066 ceph-mon[31695]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:53.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:52 smithi117 ceph-mon[27457]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:36:53.883 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:53 smithi066 ceph-mon[25398]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:53.883 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:53 smithi066 ceph-mon[31695]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:54.215 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:53 smithi117 ceph-mon[27457]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3121077944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]': finished 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.191 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3121077944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]': finished 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.192 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:54 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3121077944' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]: dispatch 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "caafa159-0619-48c7-97e3-ad1aebb5ebe0"}]': finished 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:54 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:36:56.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[25398]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:56.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi066 2024-09-10T18:36:56.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3221979813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:56.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[31695]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:56.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi066 2024-09-10T18:36:56.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:55 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3221979813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:56.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:55 smithi117 ceph-mon[27457]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:56.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:55 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi066 2024-09-10T18:36:56.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:55 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3221979813' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:36:58.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:57 smithi066 ceph-mon[25398]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:58.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:57 smithi066 ceph-mon[31695]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:36:58.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:57 smithi117 ceph-mon[27457]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:00.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:36:59 smithi066 ceph-mon[25398]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:00.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:36:59 smithi066 ceph-mon[31695]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:00.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:36:59 smithi117 ceph-mon[27457]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:01.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:37:01.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:01.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:37:01.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:01.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:37:01.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:02.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:01 smithi117 ceph-mon[27457]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:02.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:01 smithi117 ceph-mon[27457]: Deploying daemon osd.1 on smithi066 2024-09-10T18:37:02.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:01 smithi066 ceph-mon[25398]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:02.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:01 smithi066 ceph-mon[25398]: Deploying daemon osd.1 on smithi066 2024-09-10T18:37:02.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:01 smithi066 ceph-mon[31695]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:02.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:01 smithi066 ceph-mon[31695]: Deploying daemon osd.1 on smithi066 2024-09-10T18:37:04.012 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:03 smithi066 ceph-mon[25398]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:04.012 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:03 smithi066 ceph-mon[31695]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:03 smithi117 ceph-mon[27457]: pgmap v52: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:06.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:05 smithi117 ceph-mon[27457]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:06.205 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:05 smithi066 ceph-mon[31695]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:06.205 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:05 smithi066 ceph-mon[25398]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:07.178 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:07.178 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:07.180 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.180 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.180 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.180 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.180 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:06 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.211 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.212 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:06 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:07.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:07 smithi066 ceph-mon[25398]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:07.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:07 smithi066 ceph-mon[31695]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:08.074 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:07 smithi117 ceph-mon[27457]: pgmap v54: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:10.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:09 smithi066 ceph-mon[25398]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:10.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:09 smithi066 ceph-mon[25398]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:37:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:09 smithi066 ceph-mon[31695]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:09 smithi066 ceph-mon[31695]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:37:10.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:09 smithi117 ceph-mon[27457]: pgmap v55: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:10.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:09 smithi117 ceph-mon[27457]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:37:10.323 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 1 on host 'smithi066' 2024-09-10T18:37:10.991 DEBUG:teuthology.orchestra.run.smithi066:osd.1> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:37:10.993 INFO:tasks.cephadm:Deploying osd.2 on smithi066 with /dev/nvme3n1... 2024-09-10T18:37:10.994 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-10T18:37:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:37:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:37:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:11.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:10 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.288 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:11.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:37:11.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:37:11.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:11.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:11.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:11.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:10 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:12.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi066 to 7520M 2024-09-10T18:37:12.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:12.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:12.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi066 to 7520M 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:12.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi066 to 7520M 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: pgmap v57: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: from='osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/nvme3n1 2024-09-10T18:37:13.108 INFO:teuthology.orchestra.run.smithi066.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:37:13.112 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-10T18:37:13.127 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-09-10T18:37:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-09-10T18:37:13.127 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132011 s, 794 MB/s 2024-09-10T18:37:13.127 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-09-10T18:37:13.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:37:13.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:37:13.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[25398]: osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837] boot 2024-09-10T18:37:13.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[25398]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:37:13.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:13.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:37:13.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:37:13.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[31695]: osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837] boot 2024-09-10T18:37:13.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[31695]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:37:13.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:12 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:13.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:12 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:37:13.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:12 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:37:13.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:12 smithi117 ceph-mon[27457]: osd.1 [v2:172.21.15.66:6810/3509320837,v1:172.21.15.66:6811/3509320837] boot 2024-09-10T18:37:13.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:12 smithi117 ceph-mon[27457]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:37:13.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:12 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:37:13.727 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi066:/dev/nvme3n1 2024-09-10T18:37:14.022 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:14.203 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:13 smithi066 ceph-mon[25398]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:14.203 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:13 smithi066 ceph-mon[25398]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:37:14.203 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:13 smithi066 ceph-mon[31695]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:14.203 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:13 smithi066 ceph-mon[31695]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:37:14.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:13 smithi117 ceph-mon[27457]: pgmap v60: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:37:14.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:13 smithi117 ceph-mon[27457]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:37:16.049 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi066 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.050 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[31695]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi066 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.051 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.052 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:15 smithi066 ceph-mon[25398]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:37:16.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi066 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:15 smithi117 ceph-mon[27457]: Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:16.914 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.915 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:16.916 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:16.916 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:16.916 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:16.916 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:17.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:17.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:18.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:17 smithi117 ceph-mon[27457]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:18.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:17 smithi117 ceph-mon[27457]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:18.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:17 smithi117 ceph-mon[27457]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-10T18:37:18.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:17 smithi117 ceph-mon[27457]: Cluster is now healthy 2024-09-10T18:37:18.169 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[31695]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:18.169 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[31695]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[31695]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[31695]: Cluster is now healthy 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[25398]: from='client.24170 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[25398]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[25398]: Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-09-10T18:37:18.170 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:17 smithi066 ceph-mon[25398]: Cluster is now healthy 2024-09-10T18:37:20.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[25398]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:20.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]: dispatch 2024-09-10T18:37:20.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]': finished 2024-09-10T18:37:20.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[25398]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:37:20.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:20.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[31695]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:20.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]: dispatch 2024-09-10T18:37:20.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]': finished 2024-09-10T18:37:20.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[31695]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:37:20.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:20.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:19 smithi117 ceph-mon[27457]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:19 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]: dispatch 2024-09-10T18:37:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:19 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3727134672' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340a0fb8-062d-400d-a6e1-a6038be9bc42"}]': finished 2024-09-10T18:37:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:19 smithi117 ceph-mon[27457]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:37:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:21.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:20 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2949573814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:21.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:20 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2949573814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:21.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:20 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2949573814' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:22.112 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:21 smithi066 ceph-mon[25398]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:22.113 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:21 smithi066 ceph-mon[31695]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:22.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:21 smithi117 ceph-mon[27457]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:24.005 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:23 smithi066 ceph-mon[25398]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:24.006 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:23 smithi066 ceph-mon[31695]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:24.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:23 smithi117 ceph-mon[27457]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:25.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:25 smithi066 ceph-mon[25398]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:25.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:25 smithi066 ceph-mon[31695]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:26.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:25 smithi117 ceph-mon[27457]: pgmap v69: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:27.933 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:27 smithi117 ceph-mon[27457]: Deploying daemon osd.2 on smithi066 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[31695]: Deploying daemon osd.2 on smithi066 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:27.975 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:37:27.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:27.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:27.976 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:27 smithi066 ceph-mon[25398]: Deploying daemon osd.2 on smithi066 2024-09-10T18:37:29.912 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:29 smithi066 ceph-mon[25398]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:29.912 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:29 smithi066 ceph-mon[31695]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:30.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:29 smithi117 ceph-mon[27457]: pgmap v71: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:31.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:31 smithi066 ceph-mon[31695]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:31.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:31 smithi066 ceph-mon[25398]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:32.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:31 smithi117 ceph-mon[27457]: pgmap v72: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:32.900 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:32.900 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:32.900 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:32.901 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:33.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:33.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:33.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:33.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:33.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:33.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:33 smithi066 ceph-mon[25398]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:33 smithi066 ceph-mon[31695]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:34.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:33 smithi117 ceph-mon[27457]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:34.690 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 2 on host 'smithi066' 2024-09-10T18:37:34.943 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.944 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:34.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:34 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:35.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:35.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:35.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:35.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:34 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:35.345 DEBUG:teuthology.orchestra.run.smithi066:osd.2> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:37:35.347 INFO:tasks.cephadm:Deploying osd.3 on smithi066 with /dev/nvme4n1... 2024-09-10T18:37:35.348 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-10T18:37:35.665 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:36.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[25398]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:36.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi066 to 5013M 2024-09-10T18:37:36.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[25398]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:37:36.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[31695]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:36.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi066 to 5013M 2024-09-10T18:37:36.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:35 smithi066 ceph-mon[31695]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:37:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:35 smithi117 ceph-mon[27457]: pgmap v74: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:35 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi066 to 5013M 2024-09-10T18:37:36.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:35 smithi117 ceph-mon[27457]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:37:36.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:37:36.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:36.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:36.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:36.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:36.981 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2[43257]: 2024-09-10T18:37:36.877+0000 7f371a3ce640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:37:36.981 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2[43257]: 2024-09-10T18:37:36.882+0000 7f3715205640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:37.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:37.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:37.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:37.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:37.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi066 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi066 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:37:37.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:37.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:37 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:38.152 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping: /dev/nvme4n1 2024-09-10T18:37:38.152 INFO:teuthology.orchestra.run.smithi066.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:37:38.156 INFO:teuthology.orchestra.run.smithi066.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-10T18:37:38.173 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10+0 records in 2024-09-10T18:37:38.173 INFO:teuthology.orchestra.run.smithi066.stderr:10+0 records out 2024-09-10T18:37:38.173 INFO:teuthology.orchestra.run.smithi066.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0142406 s, 736 MB/s 2024-09-10T18:37:38.173 INFO:teuthology.orchestra.run.smithi066.stderr:--> Zapping successful for: 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: pgmap v76: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi066 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: from='osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:38.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:37 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:38.887 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi066:/dev/nvme4n1 2024-09-10T18:37:39.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:37:39.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:37:39.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292] boot 2024-09-10T18:37:39.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:39.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.318 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:39 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292] boot 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:39.322 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: osd.2 [v2:172.21.15.66:6818/2361686292,v1:172.21.15.66:6819/2361686292] boot 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:39.323 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:39.324 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:39 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[31695]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[31695]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:37:40.150 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[25398]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:40.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:37:40.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:37:40.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[25398]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:37:40.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:40 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:37:40.155 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:37:40.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:40 smithi117 ceph-mon[27457]: pgmap v79: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:37:40.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:40 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:37:40.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:40 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:37:40.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:40 smithi117 ceph-mon[27457]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:37:40.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:40 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:37:41.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:40 smithi066 sudo[44789]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme1n1 2024-09-10T18:37:41.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:40 smithi066 sudo[44789]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:41.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:40 smithi066 sudo[44789]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:41.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44789]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:41.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:37:41.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[31695]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:37:41.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[31695]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:41.728 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44808]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme1n1 2024-09-10T18:37:41.728 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:41.729 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44808]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:41.729 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44808]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:41.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:37:41.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[25398]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:37:41.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:41 smithi066 ceph-mon[25398]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:41.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:37:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:41 smithi117 ceph-mon[27457]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:37:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:41 smithi117 ceph-mon[27457]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:42.164 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44890]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme2n1 2024-09-10T18:37:42.164 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44890]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:42.164 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44890]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:42.164 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:41 smithi066 sudo[44890]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:42.423 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[31695]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:37:42.423 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[31695]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:37:42.423 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:42.423 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44898]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme2n1 2024-09-10T18:37:42.423 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44898]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:42.423 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44898]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:42.423 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44898]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:42.424 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[25398]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:37:42.424 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[25398]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:37:42.424 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:42.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:42 smithi117 ceph-mon[27457]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:37:42.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:42 smithi117 ceph-mon[27457]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:37:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:42.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44910]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme3n1 2024-09-10T18:37:42.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:42.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44910]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:42.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:42 smithi066 sudo[44910]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:43.646 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[44999]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme linux smart-log-add --json /dev/nvme3n1 2024-09-10T18:37:43.646 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[44999]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:43.646 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[44999]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:43.646 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[44999]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[45074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:37:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[45074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[45074]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:43 smithi066 sudo[45074]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:44.442 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 sudo[45080]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 sudo[45080]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 sudo[45080]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 sudo[45080]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.443 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.444 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:44 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: pgmap v84: 1 pgs: 1 unknown; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:44.534 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:44.792 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31011]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:37:44.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31011]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:44.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31011]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:44.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31011]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:45.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31051]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:37:45.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31051]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:45.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31051]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:45.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:44 smithi117 sudo[31051]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:45.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45140]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:37:45.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45140]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:45.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45140]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:45.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45140]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:45.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:45.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:45.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45162]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45162]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45162]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 sudo[45162]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.982 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:45.983 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:45 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='client.14298 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi066:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:46.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:37:46.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:45 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2421152795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]': finished 2024-09-10T18:37:46.971 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2421152795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]': finished 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:37:46.972 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:46 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:37:47.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2421152795' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]: dispatch 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e62a5562-9d0e-4d44-992f-b0da61fdc4fb"}]': finished 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:37:47.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:46 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:37:48.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:47 smithi117 ceph-mon[27457]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:48.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:47 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:47 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.206 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:47 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.207 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:47 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3957452076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[25398]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3957452076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[31695]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:48.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:47 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3957452076' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:37:50.218 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:49 smithi117 ceph-mon[27457]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 29 op/s 2024-09-10T18:37:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:49 smithi066 ceph-mon[31695]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 29 op/s 2024-09-10T18:37:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:49 smithi066 ceph-mon[25398]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 9.0 KiB/s rd, 457 KiB/s wr, 29 op/s 2024-09-10T18:37:52.156 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:51 smithi066 ceph-mon[31695]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-09-10T18:37:52.157 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:51 smithi066 ceph-mon[25398]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-09-10T18:37:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:51 smithi117 ceph-mon[27457]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 6.0 KiB/s rd, 305 KiB/s wr, 19 op/s 2024-09-10T18:37:53.184 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:37:53.185 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:53.185 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:37:53.185 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:53.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:37:53.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:54.151 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:53 smithi066 ceph-mon[31695]: Deploying daemon osd.3 on smithi066 2024-09-10T18:37:54.151 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:53 smithi066 ceph-mon[31695]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-09-10T18:37:54.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:53 smithi066 ceph-mon[25398]: Deploying daemon osd.3 on smithi066 2024-09-10T18:37:54.151 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:53 smithi066 ceph-mon[25398]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-09-10T18:37:54.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:53 smithi117 ceph-mon[27457]: Deploying daemon osd.3 on smithi066 2024-09-10T18:37:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:53 smithi117 ceph-mon[27457]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 4.5 KiB/s rd, 228 KiB/s wr, 14 op/s 2024-09-10T18:37:56.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:55 smithi066 ceph-mon[25398]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-09-10T18:37:56.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:55 smithi066 ceph-mon[31695]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-09-10T18:37:56.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:55 smithi117 ceph-mon[27457]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 11 op/s 2024-09-10T18:37:57.937 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:57 smithi066 ceph-mon[25398]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 168 KiB/s wr, 11 op/s 2024-09-10T18:37:57.937 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:57 smithi066 ceph-mon[31695]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 168 KiB/s wr, 11 op/s 2024-09-10T18:37:58.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:57 smithi117 ceph-mon[27457]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.3 KiB/s rd, 168 KiB/s wr, 11 op/s 2024-09-10T18:37:59.683 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.683 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.683 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.683 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 9 op/s 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.684 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 9 op/s 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.685 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:37:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 9 op/s 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:37:59.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:37:59.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:37:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:01.821 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:01 smithi066 ceph-mon[25398]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:01.821 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:01 smithi066 ceph-mon[31695]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:02.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:01 smithi117 ceph-mon[27457]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi066 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:02.961 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.962 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.962 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi066 2024-09-10T18:38:02.962 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:02 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:02.996 INFO:teuthology.orchestra.run.smithi066.stdout:Created osd(s) 3 on host 'smithi066' 2024-09-10T18:38:03.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:03.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:38:03.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi066 2024-09-10T18:38:03.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:02 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.576 DEBUG:teuthology.orchestra.run.smithi066:osd.3> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:38:03.578 INFO:tasks.cephadm:Deploying osd.4 on smithi117 with /dev/nvme1n1... 2024-09-10T18:38:03.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme1n1 2024-09-10T18:38:03.827 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:03.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:03.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:38:03.947 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi066 to 3760M 2024-09-10T18:38:03.948 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:03.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:03.949 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:03 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.978 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:38:03.688+0000 7fb947c80640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:38:03.979 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:38:03.692+0000 7fb940ab3640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi066 to 3760M 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:03.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.981 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi066 to 3760M 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:03.982 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:03 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:04.890 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:04 smithi117 ceph-mon[27457]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:38:04.890 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:04 smithi117 ceph-mon[27457]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:38:04.890 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:04.890 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:04.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[25398]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:38:04.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[25398]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:38:04.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:04.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:04.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[31695]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi066", "root=default"]}]': finished 2024-09-10T18:38:04.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[31695]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:38:04.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:04.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.530 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/nvme1n1 2024-09-10T18:38:05.530 INFO:teuthology.orchestra.run.smithi117.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:38:05.534 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme1n1 bs=1M count=10 conv=fsync 2024-09-10T18:38:05.549 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-09-10T18:38:05.549 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-09-10T18:38:05.549 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127659 s, 821 MB/s 2024-09-10T18:38:05.549 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-09-10T18:38:05.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846] boot 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:38:05.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:05 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:38:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:38:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846] boot 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: osd.3 [v2:172.21.15.66:6826/496216846,v1:172.21.15.66:6827/496216846] boot 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:38:05.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:05 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:38:06.136 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:/dev/nvme1n1 2024-09-10T18:38:06.373 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:06.790 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:06 smithi117 ceph-mon[27457]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:38:06.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:06 smithi066 ceph-mon[31695]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:38:06.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:06 smithi066 ceph-mon[25398]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:08.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:07 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:08.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:08.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:08.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:07 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:09.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:08 smithi117 ceph-mon[27457]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:09.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:08 smithi066 ceph-mon[31695]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:09.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:08 smithi066 ceph-mon[25398]: from='client.24205 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme1n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:10.295 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:09 smithi117 ceph-mon[27457]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:10.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:09 smithi066 ceph-mon[31695]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:10.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:09 smithi066 ceph-mon[25398]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/3527742724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]': finished 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:38:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/3527742724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]': finished 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:38:12.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/3527742724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]: dispatch 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1a94533a-e50d-4f00-a683-a14921c963f7"}]': finished 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:38:12.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:14.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:13 smithi117 ceph-mon[27457]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:14.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:13 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/3626565333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:14.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:13 smithi066 ceph-mon[31695]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:14.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:13 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/3626565333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:14.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:13 smithi066 ceph-mon[25398]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:14.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:13 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/3626565333' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:16.257 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:16 smithi117 ceph-mon[27457]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:16.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:16 smithi066 ceph-mon[25398]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:16.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:16 smithi066 ceph-mon[31695]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:18.304 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:18 smithi117 ceph-mon[27457]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:18.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:18 smithi066 ceph-mon[25398]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:18.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:18 smithi066 ceph-mon[31695]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:19.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:38:19.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:19.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:38:19.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:19.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:38:19.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:20.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:20 smithi117 ceph-mon[27457]: Deploying daemon osd.4 on smithi117 2024-09-10T18:38:20.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:20 smithi117 ceph-mon[27457]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:20.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.188 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.189 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[31695]: Deploying daemon osd.4 on smithi117 2024-09-10T18:38:20.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[31695]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:20.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[25398]: Deploying daemon osd.4 on smithi117 2024-09-10T18:38:20.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[25398]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:20.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:20.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:22.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:22 smithi117 ceph-mon[27457]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:22.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:22 smithi066 ceph-mon[31695]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:22.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:22 smithi066 ceph-mon[25398]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:24 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:24.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:24 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:26.074 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 4 on host 'smithi117' 2024-09-10T18:38:26.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:26 smithi117 ceph-mon[27457]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:26.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:26 smithi066 ceph-mon[25398]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:26.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:26 smithi066 ceph-mon[31695]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:26.709 DEBUG:teuthology.orchestra.run.smithi117:osd.4> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:38:26.711 INFO:tasks.cephadm:Deploying osd.5 on smithi117 with /dev/nvme2n1... 2024-09-10T18:38:26.711 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme2n1 2024-09-10T18:38:27.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:27 smithi117 ceph-mon[27457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:27.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:27.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:27.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:27.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:27.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:27 smithi066 ceph-mon[25398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:38:27.643 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi117 to 16064M 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.318 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.318 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:38:28.103+0000 7ff2146fb640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:38:28.318 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:38:28 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:38:28.109+0000 7ff211536640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:38:28.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi117 to 16064M 2024-09-10T18:38:28.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:28.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:38:28.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi117 to 16064M 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:28.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:28.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:28 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:29.287 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:29 smithi117 ceph-mon[27457]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:29.287 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:29 smithi117 ceph-mon[27457]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:38:29.288 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.288 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.288 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:29 smithi117 ceph-mon[27457]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:29.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[31695]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:29.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[31695]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:38:29.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[31695]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[25398]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[25398]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:38:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:29 smithi066 ceph-mon[25398]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:38:30.148 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:30 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:38:30.148 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:30 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:38:30.148 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:30 smithi117 ceph-mon[27457]: osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432] boot 2024-09-10T18:38:30.148 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:30 smithi117 ceph-mon[27457]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:38:30.148 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:30 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:30.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:38:30.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:38:30.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[25398]: osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432] boot 2024-09-10T18:38:30.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[25398]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:38:30.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:30.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:38:30.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:38:30.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[31695]: osd.4 [v2:172.21.15.117:6800/5085432,v1:172.21.15.117:6801/5085432] boot 2024-09-10T18:38:30.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[31695]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:38:30.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:30 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:38:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:31 smithi117 ceph-mon[27457]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:38:31.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:31 smithi117 ceph-mon[27457]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:31.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:31 smithi117 ceph-mon[27457]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:38:31.470 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/nvme2n1 2024-09-10T18:38:31.470 INFO:teuthology.orchestra.run.smithi117.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:38:31.475 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme2n1 bs=1M count=10 conv=fsync 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[31695]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[31695]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[31695]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[25398]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[25398]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:31.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:31 smithi066 ceph-mon[25398]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:38:31.490 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-09-10T18:38:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-09-10T18:38:31.490 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127144 s, 825 MB/s 2024-09-10T18:38:31.490 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-09-10T18:38:32.094 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:/dev/nvme2n1 2024-09-10T18:38:32.342 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:33.433 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:33 smithi117 ceph-mon[27457]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:38:33.433 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:33 smithi117 ceph-mon[27457]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:33.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:33 smithi066 ceph-mon[31695]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:38:33.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:33 smithi066 ceph-mon[31695]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:33.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:33 smithi066 ceph-mon[25398]: osdmap e33: 5 total, 5 up, 5 in 2024-09-10T18:38:33.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:33 smithi066 ceph-mon[25398]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:34.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:34 smithi117 ceph-mon[27457]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T18:38:34.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:34 smithi066 ceph-mon[31695]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T18:38:34.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:34 smithi066 ceph-mon[25398]: osdmap e34: 5 total, 5 up, 5 in 2024-09-10T18:38:35.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:35 smithi117 ceph-mon[27457]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:35.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:35.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:35.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:35 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:35.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:35 smithi117 ceph-mon[27457]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-10T18:38:35.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[31695]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:35.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:35.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:35.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:35.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[31695]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-10T18:38:35.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[25398]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme2n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:38:35.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:35.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:35.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:35.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:35 smithi066 ceph-mon[25398]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-10T18:38:36.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:36 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/1011377007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:36 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:36 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]': finished 2024-09-10T18:38:36.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:36 smithi117 ceph-mon[27457]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:38:36.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:36 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/1011377007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]': finished 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[31695]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/1011377007' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]: dispatch 2024-09-10T18:38:36.950 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "372defb5-3bca-4453-ab60-88741d9ab57b"}]': finished 2024-09-10T18:38:36.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[25398]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:38:36.951 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:36 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:37.667 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:37 smithi117 ceph-mon[27457]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T18:38:37.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:37 smithi066 ceph-mon[25398]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T18:38:37.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:37 smithi066 ceph-mon[31695]: pgmap v125: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-09-10T18:38:38.601 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:38 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/1112384590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:38.601 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:38 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/1112384590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:38.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:38 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/1112384590' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:38:39.618 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:39 smithi066 ceph-mon[31695]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T18:38:39.618 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:39 smithi066 ceph-mon[25398]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T18:38:39.806 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:39 smithi117 ceph-mon[27457]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-10T18:38:41.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.197 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:41 smithi117 ceph-mon[27457]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:41.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[25398]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:41.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:41.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:41 smithi066 ceph-mon[31695]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:38:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[25398]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:38:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[25398]: Deploying daemon osd.5 on smithi117 2024-09-10T18:38:43.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[31695]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:43.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:38:43.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:43.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:43 smithi066 ceph-mon[31695]: Deploying daemon osd.5 on smithi117 2024-09-10T18:38:44.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:43 smithi117 ceph-mon[27457]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:44.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:43 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:38:44.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:43 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:44.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:43 smithi117 ceph-mon[27457]: Deploying daemon osd.5 on smithi117 2024-09-10T18:38:45.770 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:45 smithi117 ceph-mon[27457]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:45.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:45 smithi066 ceph-mon[25398]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:45.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:45 smithi066 ceph-mon[31695]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:47.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:47 smithi117 ceph-mon[27457]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:47.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:47 smithi066 ceph-mon[25398]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:47.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:47 smithi066 ceph-mon[31695]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:49.103 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi117 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:49.104 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:49.105 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.105 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:48 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi117 2024-09-10T18:38:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi117 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:48 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:49 smithi117 ceph-mon[27457]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.203 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[31695]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[25398]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:52.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:51 smithi117 ceph-mon[27457]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:52.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:51 smithi117 ceph-mon[27457]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:52.204 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:51 smithi117 ceph-mon[27457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:52.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[25398]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:52.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[25398]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:52.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[25398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:52.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[31695]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:52.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[31695]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:52.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:51 smithi066 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:38:53.150 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:52 smithi117 ceph-mon[27457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:38:53.150 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:52 smithi117 ceph-mon[27457]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:38:53.150 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:53.150 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:52 smithi117 ceph-mon[27457]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.150 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:52 smithi117 ceph-mon[27457]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[25398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:38:53.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[25398]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:38:53.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:53.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[25398]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[25398]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:38:53.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[31695]: osdmap e36: 6 total, 5 up, 6 in 2024-09-10T18:38:53.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:53.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[31695]: from='osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:52 smithi066 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:38:53.438 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 5 on host 'smithi117' 2024-09-10T18:38:54.053 DEBUG:teuthology.orchestra.run.smithi117:osd.5> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:38:54.055 INFO:tasks.cephadm:Deploying osd.6 on smithi117 with /dev/nvme3n1... 2024-09-10T18:38:54.056 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme3n1 2024-09-10T18:38:54.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608] boot 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:54.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:54.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:54.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608] boot 2024-09-10T18:38:54.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:38:54.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:53 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: osd.5 [v2:172.21.15.117:6808/1012691608,v1:172.21.15.117:6809/1012691608] boot 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:38:54.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:53 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:38:54.330 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:55.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:54 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:38:55.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:54 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:38:55.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:54 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi117 to 8032M 2024-09-10T18:38:55.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:54 smithi117 ceph-mon[27457]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:38:55.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:38:55.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:38:55.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi117 to 8032M 2024-09-10T18:38:55.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[31695]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:38:55.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:38:55.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:38:55.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi117 to 8032M 2024-09-10T18:38:55.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:54 smithi066 ceph-mon[25398]: osdmap e39: 6 total, 6 up, 6 in 2024-09-10T18:38:56.137 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/nvme3n1 2024-09-10T18:38:56.137 INFO:teuthology.orchestra.run.smithi117.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:38:56.142 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme3n1 bs=1M count=10 conv=fsync 2024-09-10T18:38:56.157 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-09-10T18:38:56.157 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-09-10T18:38:56.157 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128426 s, 816 MB/s 2024-09-10T18:38:56.157 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-09-10T18:38:56.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[31695]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:56.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[31695]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:56.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[31695]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T18:38:56.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[25398]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:56.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[25398]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:56.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:55 smithi066 ceph-mon[25398]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T18:38:56.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:55 smithi117 ceph-mon[27457]: pgmap v137: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:56.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:55 smithi117 ceph-mon[27457]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:38:56.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:55 smithi117 ceph-mon[27457]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T18:38:56.773 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:/dev/nvme3n1 2024-09-10T18:38:57.011 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:38:58.206 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:57 smithi066 ceph-mon[31695]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:58.206 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:57 smithi066 ceph-mon[31695]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T18:38:58.207 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:57 smithi066 ceph-mon[25398]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:58.207 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:57 smithi066 ceph-mon[25398]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T18:38:58.267 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:57 smithi117 ceph-mon[27457]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:38:58.267 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:57 smithi117 ceph-mon[27457]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T18:38:59.188 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[31695]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T18:38:59.188 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:59.188 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:59.189 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:59.189 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[25398]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T18:38:59.189 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:59.189 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:59.189 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:58 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:38:59.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:58 smithi117 ceph-mon[27457]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T18:38:59.306 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:58 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:38:59.307 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:58 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:38:59.307 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:58 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:00.220 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:59 smithi066 ceph-mon[31695]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:00.220 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:38:59 smithi066 ceph-mon[31695]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:00.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:59 smithi066 ceph-mon[25398]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:00.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:38:59 smithi066 ceph-mon[25398]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:00.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:59 smithi117 ceph-mon[27457]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:00.312 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:38:59 smithi117 ceph-mon[27457]: from='client.24259 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme3n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/4245981993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]': finished 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T18:39:01.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:01 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/4245981993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]': finished 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T18:39:01.634 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/4245981993' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]: dispatch 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e25289b-3df3-477a-a98e-bc6ecf4fd7fc"}]': finished 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: osdmap e43: 7 total, 6 up, 7 in 2024-09-10T18:39:01.635 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:01 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:02.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:02 smithi117 ceph-mon[27457]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:39:02.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:02 smithi117 ceph-mon[27457]: Cluster is now healthy 2024-09-10T18:39:02.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:02 smithi066 ceph-mon[25398]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:39:02.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:02 smithi066 ceph-mon[25398]: Cluster is now healthy 2024-09-10T18:39:02.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:02 smithi066 ceph-mon[31695]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:39:02.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:02 smithi066 ceph-mon[31695]: Cluster is now healthy 2024-09-10T18:39:03.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:03 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/2069275130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:03.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:03 smithi117 ceph-mon[27457]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:39:03.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:03 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/2069275130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:03.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:03 smithi066 ceph-mon[25398]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:39:03.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:03 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/2069275130' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:03.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:03 smithi066 ceph-mon[31695]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:39:05.971 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:05 smithi117 ceph-mon[27457]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T18:39:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:05 smithi066 ceph-mon[25398]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T18:39:05.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:05 smithi066 ceph-mon[31695]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-09-10T18:39:07.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:07 smithi066 ceph-mon[31695]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:39:07.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:07 smithi066 ceph-mon[25398]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:39:08.059 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:07 smithi117 ceph-mon[27457]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: Deploying daemon osd.6 on smithi117 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi117 2024-09-10T18:39:08.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: Deploying daemon osd.6 on smithi117 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi117 2024-09-10T18:39:08.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: Deploying daemon osd.6 on smithi117 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi117 2024-09-10T18:39:09.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:10.216 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:10 smithi117 ceph-mon[27457]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:39:10.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:10 smithi066 ceph-mon[25398]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:39:10.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:10 smithi066 ceph-mon[31695]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:39:12.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:12 smithi117 ceph-mon[27457]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:12.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:12 smithi066 ceph-mon[25398]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:12.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:12 smithi066 ceph-mon[31695]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:14.277 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:14.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.278 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:14 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:14.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:14.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:14 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:15.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:15 smithi117 ceph-mon[27457]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:15.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:15 smithi066 ceph-mon[25398]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:15.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:15 smithi066 ceph-mon[31695]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:16.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:16 smithi117 ceph-mon[27457]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:16.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:16 smithi117 ceph-mon[27457]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:16.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:16 smithi066 ceph-mon[25398]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:16.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:16 smithi066 ceph-mon[25398]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:16.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:16 smithi066 ceph-mon[31695]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:16.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:16 smithi066 ceph-mon[31695]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:39:17.298 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 6 on host 'smithi117' 2024-09-10T18:39:17.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:17.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:39:17.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T18:39:17.907 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:17.908 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:17.909 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.932 DEBUG:teuthology.orchestra.run.smithi117:osd.6> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:39:17.934 INFO:tasks.cephadm:Deploying osd.7 on smithi117 with /dev/nvme4n1... 2024-09-10T18:39:17.934 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/nvme4n1 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:17.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: osdmap e44: 7 total, 6 up, 7 in 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:17.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:17.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:17.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:17.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:17 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:18.209 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi117 to 5354M 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872] boot 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T18:39:18.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi117 to 5354M 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872] boot 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T18:39:18.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:18 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:19.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi117 to 5354M 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: osdmap e45: 7 total, 6 up, 7 in 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: osd.6 [v2:172.21.15.117:6816/1071681872,v1:172.21.15.117:6817/1071681872] boot 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: osdmap e46: 7 total, 7 up, 7 in 2024-09-10T18:39:19.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:18 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:39:19.972 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:39:19.972 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:39:19.972 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[31695]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:19.973 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[31695]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T18:39:19.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:39:19.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:39:19.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[25398]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:19.973 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:19 smithi066 ceph-mon[25398]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T18:39:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/nvme4n1 2024-09-10T18:39:20.027 INFO:teuthology.orchestra.run.smithi117.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-09-10T18:39:20.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:19 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:39:20.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:19 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:39:20.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:19 smithi117 ceph-mon[27457]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:39:20.028 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:19 smithi117 ceph-mon[27457]: osdmap e47: 7 total, 7 up, 7 in 2024-09-10T18:39:20.031 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-09-10T18:39:20.046 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-09-10T18:39:20.046 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-09-10T18:39:20.046 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129484 s, 810 MB/s 2024-09-10T18:39:20.046 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-09-10T18:39:20.670 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi117:/dev/nvme4n1 2024-09-10T18:39:20.917 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:39:21.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[25398]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[25398]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[31695]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:21.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:21 smithi066 ceph-mon[31695]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T18:39:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:21 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:21 smithi117 ceph-mon[27457]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:21.734 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:21 smithi117 ceph-mon[27457]: osdmap e48: 7 total, 7 up, 7 in 2024-09-10T18:39:23.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T18:39:23.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:23.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:23.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:39:23.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:23 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: osdmap e49: 7 total, 7 up, 7 in 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: pgmap v162: 1 pgs: 1 remapped+peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:39:23.793 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:39:23.794 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:23 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:24.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:24 smithi117 ceph-mon[27457]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T18:39:24.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:24 smithi066 ceph-mon[25398]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T18:39:24.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:24 smithi066 ceph-mon[31695]: osdmap e50: 7 total, 7 up, 7 in 2024-09-10T18:39:25.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T18:39:25.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/2551205818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]': finished 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/2551205818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]': finished 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:39:25.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:25 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:25.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-09-10T18:39:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/2551205818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]: dispatch 2024-09-10T18:39:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b179067d-9cd3-40b6-978c-ae59ec89af6b"}]': finished 2024-09-10T18:39:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: osdmap e51: 8 total, 7 up, 8 in 2024-09-10T18:39:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:25 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:26.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:26 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/2854393392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:26.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:26 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/2854393392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:26.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:26 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/2854393392' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:39:27.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:27 smithi066 ceph-mon[25398]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T18:39:27.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:27 smithi066 ceph-mon[31695]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T18:39:27.735 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:27 smithi117 ceph-mon[27457]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi117 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:29 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi117 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi117 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 99 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-09-10T18:39:29.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:29 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:31.722 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:31 smithi117 ceph-mon[27457]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T18:39:31.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:31 smithi066 ceph-mon[31695]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T18:39:31.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:31 smithi066 ceph-mon[25398]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-09-10T18:39:32.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:39:32.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:32.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[31695]: Deploying daemon osd.7 on smithi117 2024-09-10T18:39:32.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:39:32.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:32.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:32 smithi066 ceph-mon[25398]: Deploying daemon osd.7 on smithi117 2024-09-10T18:39:32.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:39:32.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:32.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:32 smithi117 ceph-mon[27457]: Deploying daemon osd.7 on smithi117 2024-09-10T18:39:33.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:33 smithi066 ceph-mon[31695]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:33.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:33 smithi066 ceph-mon[25398]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:33.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:33 smithi117 ceph-mon[27457]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:35.641 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:35 smithi117 ceph-mon[27457]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:35.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:35 smithi066 ceph-mon[25398]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:35.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:35 smithi066 ceph-mon[31695]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:37.638 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:37 smithi117 ceph-mon[27457]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:37.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:37 smithi066 ceph-mon[25398]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:37.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:37 smithi066 ceph-mon[31695]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:38.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:38.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:38.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:38.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:38 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:38.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:38.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:38.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:38.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:38 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:40.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:39 smithi117 ceph-mon[27457]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:40.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:39 smithi066 ceph-mon[25398]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:40.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:39 smithi066 ceph-mon[31695]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:41.551 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 7 on host 'smithi117' 2024-09-10T18:39:41.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:41.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:41 smithi117 ceph-mon[27457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:41.816 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[25398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:41.817 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:41.818 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:41 smithi066 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:39:42.184 DEBUG:teuthology.orchestra.run.smithi117:osd.7> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:39:42.186 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:39:42.187 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:39:42.554 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:42.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: Adjusting osd_memory_target on smithi117 to 4016M 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:42.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[25398]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:42.730 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: Adjusting osd_memory_target on smithi117 to 4016M 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:39:42.731 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T18:39:42.732 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:42.732 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:42.732 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:42 smithi066 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:42.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: Adjusting osd_memory_target on smithi117 to 4016M 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:42.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: osdmap e52: 8 total, 7 up, 8 in 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:42.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-mon[27457]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-09-10T18:39:43.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:39:42.961+0000 7f27286c6640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:39:43.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:39:42 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:39:42.971+0000 7f27234fd640 -1 osd.7 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:39:44.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[25398]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:44.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[25398]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:44.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[25398]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[31695]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[31695]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[31695]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:43 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.239 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:39:44.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:43 smithi117 ceph-mon[27457]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:44.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:43 smithi117 ceph-mon[27457]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-09-10T18:39:44.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:43 smithi117 ceph-mon[27457]: osdmap e53: 8 total, 7 up, 8 in 2024-09-10T18:39:44.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:43 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:43 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:44.885 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1725993583,"num_in_osds":8,"osd_in_since":1725993565,"num_remapped_pgs":1} 2024-09-10T18:39:44.885 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:39:45.149 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:45.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: purged_snaps scrub starts 2024-09-10T18:39:45.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: purged_snaps scrub ok 2024-09-10T18:39:45.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576] boot 2024-09-10T18:39:45.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2429037797' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: purged_snaps scrub starts 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: purged_snaps scrub ok 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576] boot 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:45.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:44 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2429037797' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:39:45.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: purged_snaps scrub starts 2024-09-10T18:39:45.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: purged_snaps scrub ok 2024-09-10T18:39:45.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: osd.7 [v2:172.21.15.117:6824/306391576,v1:172.21.15.117:6825/306391576] boot 2024-09-10T18:39:45.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:39:45.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:39:45.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:44 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2429037797' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:39:46.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:45 smithi066 ceph-mon[31695]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:46.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:45 smithi066 ceph-mon[31695]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:39:46.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:45 smithi066 ceph-mon[25398]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:46.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:45 smithi066 ceph-mon[25398]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:39:46.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:45 smithi117 ceph-mon[27457]: pgmap v179: 1 pgs: 1 peering; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:39:46.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:45 smithi117 ceph-mon[27457]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:39:46.884 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:39:46.884 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":56,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","created":"2024-09-10T18:34:27.706202+0000","modified":"2024-09-10T18:39:45.965433+0000","last_up_change":"2024-09-10T18:39:43.955218+0000","last_in_change":"2024-09-10T18:39:25.004567+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:37:39.052063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"686cf549-884c-444b-a3c4-d1f69af43911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6803","nonce":4045628832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6805","nonce":4045628832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6809","nonce":4045628832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6807","nonce":4045628832}]},"public_addr":"172.21.15.66:6803/4045628832","cluster_addr":"172.21.15.66:6805/4045628832","heartbeat_back_addr":"172.21.15.66:6809/4045628832","heartbeat_front_addr":"172.21.15.66:6807/4045628832","state":["exists","up"]},{"osd":1,"uuid":"caafa159-0619-48c7-97e3-ad1aebb5ebe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6811","nonce":3509320837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6813","nonce":3509320837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6817","nonce":3509320837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6815","nonce":3509320837}]},"public_addr":"172.21.15.66:6811/3509320837","cluster_addr":"172.21.15.66:6813/3509320837","heartbeat_back_addr":"172.21.15.66:6817/3509320837","heartbeat_front_addr":"172.21.15.66:6815/3509320837","state":["exists","up"]},{"osd":2,"uuid":"340a0fb8-062d-400d-a6e1-a6038be9bc42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6819","nonce":2361686292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6821","nonce":2361686292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6825","nonce":2361686292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6823","nonce":2361686292}]},"public_addr":"172.21.15.66:6819/2361686292","cluster_addr":"172.21.15.66:6821/2361686292","heartbeat_back_addr":"172.21.15.66:6825/2361686292","heartbeat_front_addr":"172.21.15.66:6823/2361686292","state":["exists","up"]},{"osd":3,"uuid":"e62a5562-9d0e-4d44-992f-b0da61fdc4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6827","nonce":496216846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6829","nonce":496216846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6833","nonce":496216846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6831","nonce":496216846}]},"public_addr":"172.21.15.66:6827/496216846","cluster_addr":"172.21.15.66:6829/496216846","heartbeat_back_addr":"172.21.15.66:6833/496216846","heartbeat_front_addr":"172.21.15.66:6831/496216846","state":["exists","up"]},{"osd":4,"uuid":"1a94533a-e50d-4f00-a683-a14921c963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6801","nonce":5085432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6803","nonce":5085432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6807","nonce":5085432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6805","nonce":5085432}]},"public_addr":"172.21.15.117:6801/5085432","cluster_addr":"172.21.15.117:6803/5085432","heartbeat_back_addr":"172.21.15.117:6807/5085432","heartbeat_front_addr":"172.21.15.117:6805/5085432","state":["exists","up"]},{"osd":5,"uuid":"372defb5-3bca-4453-ab60-88741d9ab57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6809","nonce":1012691608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6811","nonce":1012691608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6815","nonce":1012691608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6813","nonce":1012691608}]},"public_addr":"172.21.15.117:6809/1012691608","cluster_addr":"172.21.15.117:6811/1012691608","heartbeat_back_addr":"172.21.15.117:6815/1012691608","heartbeat_front_addr":"172.21.15.117:6813/1012691608","state":["exists","up"]},{"osd":6,"uuid":"9e25289b-3df3-477a-a98e-bc6ecf4fd7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6817","nonce":1071681872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6819","nonce":1071681872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6823","nonce":1071681872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6821","nonce":1071681872}]},"public_addr":"172.21.15.117:6817/1071681872","cluster_addr":"172.21.15.117:6819/1071681872","heartbeat_back_addr":"172.21.15.117:6823/1071681872","heartbeat_front_addr":"172.21.15.117:6821/1071681872","state":["exists","up"]},{"osd":7,"uuid":"b179067d-9cd3-40b6-978c-ae59ec89af6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6825","nonce":306391576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6827","nonce":306391576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6831","nonce":306391576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6829","nonce":306391576}]},"public_addr":"172.21.15.117:6825/306391576","cluster_addr":"172.21.15.117:6827/306391576","heartbeat_back_addr":"172.21.15.117:6831/306391576","heartbeat_front_addr":"172.21.15.117:6829/306391576","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:46.756861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:10.614445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:35.901695+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:02.748776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:27.460845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:52.832490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:17.652789+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:42.007242+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[{"pgid":"1.0","osds":[0,6]}],"primary_temp":[],"blocklist":{"172.21.15.66:0/3684972778":"2024-09-11T18:35:18.553127+0000","172.21.15.66:6800/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/3287086836":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/2654732106":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/2959308788":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/1207688396":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/115463389":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6801/80391987":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6801/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:6800/80391987":"2024-09-11T18:34:55.725213+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:39:47.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:46 smithi066 ceph-mon[25398]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:39:47.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:46 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2388057555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:47.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:46 smithi066 ceph-mon[31695]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:39:47.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:46 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2388057555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:47.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:46 smithi117 ceph-mon[27457]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:39:47.263 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:46 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2388057555' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:39:47.458 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:37:39.052063+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:39:47.459 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:39:47.709 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:39:48.139 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:47 smithi066 ceph-mon[25398]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:48.139 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:47 smithi066 ceph-mon[25398]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:39:48.139 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:47 smithi066 ceph-mon[31695]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:48.140 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:47 smithi066 ceph-mon[31695]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:39:48.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:47 smithi117 ceph-mon[27457]: pgmap v182: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:48.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:47 smithi117 ceph-mon[27457]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:39:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:39:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.231 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.232 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:49 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:49.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:49.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:49.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:49 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:49.360 INFO:teuthology.orchestra.run.smithi066.stdout:pg_num: 1 2024-09-10T18:39:50.072 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi066 2024-09-10T18:39:50.072 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi066=foo.a' 2024-09-10T18:39:50.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:50 smithi117 ceph-mon[27457]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:50 smithi117 ceph-mon[27457]: Detected new or changed devices on smithi117 2024-09-10T18:39:50.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:50 smithi117 ceph-mon[27457]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:50 smithi117 ceph-mon[27457]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:50 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1725488969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:39:50.321 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:39:50.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[31695]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[31695]: Detected new or changed devices on smithi117 2024-09-10T18:39:50.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[31695]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[31695]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1725488969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:39:50.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[25398]: pgmap v185: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[25398]: Detected new or changed devices on smithi117 2024-09-10T18:39:50.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[25398]: pgmap v186: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[25398]: pgmap v187: 1 pgs: 1 peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:39:50.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:50 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1725488969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:39:51.968 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:39:52.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:52 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.418 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:52.419 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:52 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:52.514 DEBUG:teuthology.orchestra.run.smithi066:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:39:52.517 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi117 2024-09-10T18:39:52.517 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:39:52.769 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:39:53.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:53 smithi117 ceph-mon[27457]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi066=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:53.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:53 smithi117 ceph-mon[27457]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:53.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:53 smithi117 ceph-mon[27457]: Deploying daemon rgw.foo.a on smithi066 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[31695]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi066=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[31695]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[31695]: Deploying daemon rgw.foo.a on smithi066 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[25398]: from='client.24331 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi066=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[25398]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:53.374 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:53 smithi066 ceph-mon[25398]: Deploying daemon rgw.foo.a on smithi066 2024-09-10T18:39:53.978 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:53 smithi066 systemd[1]: Starting Ceph rgw.foo.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:39:54.095 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:54 smithi117 ceph-mon[27457]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T18:39:54.320 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:54 smithi066 ceph-mon[25398]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T18:39:54.320 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:54 smithi066 ceph-mon[31695]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-10T18:39:54.320 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 podman[50513]: 2024-09-10 18:39:54.032824077 +0000 UTC m=+0.108187908 container create e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-10T18:39:54.320 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 podman[50513]: 2024-09-10 18:39:53.94093931 +0000 UTC m=+0.016303142 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:39:54.321 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 podman[50513]: 2024-09-10 18:39:54.243667105 +0000 UTC m=+0.319030936 container init e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:39:54.321 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 podman[50513]: 2024-09-10 18:39:54.24667674 +0000 UTC m=+0.322040583 container start e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:39:54.321 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 bash[50513]: e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 2024-09-10T18:39:54.728 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:39:54 smithi066 systemd[1]: Started Ceph rgw.foo.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:39:55.154 INFO:teuthology.orchestra.run.smithi117.stderr:pool 'datapool' created 2024-09-10T18:39:55.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:55 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/2634041238' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:55 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:55 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/2634041238' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:55 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:55 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/2634041238' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:55 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:39:55.798 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:39:56.229 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:39:56.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:56.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[31695]: Checking dashboard <-> RGW credentials 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:56.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:56 smithi066 ceph-mon[25398]: Checking dashboard <-> RGW credentials 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:39:56.493 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: Saving service rgw.foo.a spec with placement smithi066=foo.a;count:1 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:39:56.494 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:56 smithi117 ceph-mon[27457]: Checking dashboard <-> RGW credentials 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[31695]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[31695]: pgmap v193: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[25398]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:39:57.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:57 smithi066 ceph-mon[25398]: pgmap v193: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:39:57.496 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:57 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:39:57.496 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:57 smithi117 ceph-mon[27457]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:39:57.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:57 smithi117 ceph-mon[27457]: pgmap v193: 36 pgs: 4 creating+peering, 31 unknown, 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-10T18:39:58.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:58.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:58 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:58.497 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:58 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: pgmap v196: 68 pgs: 9 creating+peering, 51 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/1424512054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:59.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: pgmap v196: 68 pgs: 9 creating+peering, 51 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/1424512054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:39:59 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:59.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: pgmap v196: 68 pgs: 9 creating+peering, 51 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/1424512054' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:39:59.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:39:59 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:40:00.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:00.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: application not enabled on pool 'datapool' 2024-09-10T18:40:00.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[25398]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: application not enabled on pool 'datapool' 2024-09-10T18:40:00.481 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:00 smithi066 ceph-mon[31695]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2136933308' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3385529256' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:00.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T18:40:00.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: application not enabled on pool 'datapool' 2024-09-10T18:40:00.567 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:00 smithi117 ceph-mon[27457]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T18:40:01.467 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:01 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.467 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:01 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.467 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:01 smithi117 ceph-mon[27457]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:01.467 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:01 smithi117 ceph-mon[27457]: pgmap v199: 100 pgs: 13 creating+peering, 35 unknown, 52 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[31695]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[31695]: pgmap v199: 100 pgs: 13 creating+peering, 35 unknown, 52 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:40:01.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[25398]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:40:01.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:01 smithi066 ceph-mon[25398]: pgmap v199: 100 pgs: 13 creating+peering, 35 unknown, 52 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.5 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T18:40:01.977 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.117 --placement '1;smithi117=iscsi.a' 2024-09-10T18:40:02.231 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:02.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:40:02.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: Cluster is now healthy 2024-09-10T18:40:02.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:02.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:02 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: Cluster is now healthy 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: Cluster is now healthy 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:02.479 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:02 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:03 smithi117 ceph-mon[27457]: pgmap v202: 132 pgs: 9 creating+peering, 56 unknown, 67 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:40:03.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[31695]: pgmap v202: 132 pgs: 9 creating+peering, 56 unknown, 67 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:40:03.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:40:03.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1194419367' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/93027631' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:40:03.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:03 smithi066 ceph-mon[25398]: pgmap v202: 132 pgs: 9 creating+peering, 56 unknown, 67 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:40:04.093 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:40:04.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: Checking dashboard <-> RGW credentials 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:40:04.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:04 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: Checking dashboard <-> RGW credentials 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.480 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: Checking dashboard <-> RGW credentials 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.481 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:40:04.482 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:04 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:04.717 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:40:04.717 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:40:04.717 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:40:04.750 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:40:04.750 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:40:04.779 DEBUG:teuthology.orchestra.run.smithi117:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:40:04.821 INFO:tasks.cephadm:Adding prometheus.a on smithi117 2024-09-10T18:40:04.821 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi117=a' 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[25398]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.117", "placement": "1;smithi117=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[25398]: Saving service iscsi.datapool spec with placement smithi117=iscsi.a;count:1 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[25398]: Deploying daemon iscsi.iscsi.a on smithi117 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[25398]: pgmap v204: 132 pgs: 4 creating+peering, 13 unknown, 115 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.2 KiB/s wr, 159 op/s 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[31695]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.117", "placement": "1;smithi117=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:05.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[31695]: Saving service iscsi.datapool spec with placement smithi117=iscsi.a;count:1 2024-09-10T18:40:05.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[31695]: Deploying daemon iscsi.iscsi.a on smithi117 2024-09-10T18:40:05.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:05 smithi066 ceph-mon[31695]: pgmap v204: 132 pgs: 4 creating+peering, 13 unknown, 115 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.2 KiB/s wr, 159 op/s 2024-09-10T18:40:05.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:05 smithi117 ceph-mon[27457]: from='client.24436 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.117", "placement": "1;smithi117=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:05.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:05 smithi117 ceph-mon[27457]: Saving service iscsi.datapool spec with placement smithi117=iscsi.a;count:1 2024-09-10T18:40:05.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:05 smithi117 ceph-mon[27457]: Deploying daemon iscsi.iscsi.a on smithi117 2024-09-10T18:40:05.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:05 smithi117 ceph-mon[27457]: pgmap v204: 132 pgs: 4 creating+peering, 13 unknown, 115 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.2 KiB/s wr, 159 op/s 2024-09-10T18:40:05.582 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:06.566 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:06 smithi117 systemd[1]: Starting Ceph iscsi.iscsi.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:40:07.654 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 podman[46749]: 2024-09-10 18:40:07.311882724 +0000 UTC m=+0.200517499 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:40:07.654 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 podman[46749]: 2024-09-10 18:40:07.433757595 +0000 UTC m=+0.322392369 container create 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T18:40:07.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:07 smithi117 ceph-mon[27457]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-10T18:40:07.915 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:07 smithi117 ceph-mon[27457]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:07.915 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled prometheus update... 2024-09-10T18:40:07.916 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 podman[46749]: 2024-09-10 18:40:07.870270982 +0000 UTC m=+0.758905755 container init 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T18:40:07.916 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 podman[46749]: 2024-09-10 18:40:07.87403318 +0000 UTC m=+0.762667954 container start 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T18:40:07.916 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[46797]: log file path now is '/var/log/tcmu-runner.log' 2024-09-10T18:40:07.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:07 smithi066 ceph-mon[25398]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-10T18:40:07.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:07 smithi066 ceph-mon[25398]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:07.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:07 smithi066 ceph-mon[31695]: pgmap v205: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 65 KiB/s rd, 5.2 KiB/s wr, 156 op/s 2024-09-10T18:40:07.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:07 smithi066 ceph-mon[31695]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:40:08.316 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:07 smithi117 bash[46749]: 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 2024-09-10T18:40:08.316 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:08 smithi117 podman[46815]: 2024-09-10 18:40:08.006898438 +0000 UTC m=+0.447186057 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:40:08.627 DEBUG:teuthology.orchestra.run.smithi117:prometheus.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:40:08.630 INFO:tasks.cephadm:Adding node-exporter.a on smithi066 2024-09-10T18:40:08.630 INFO:tasks.cephadm:Adding node-exporter.b on smithi117 2024-09-10T18:40:08.630 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi066=a;smithi117=b' 2024-09-10T18:40:08.875 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:08 smithi117 podman[46815]: 2024-09-10 18:40:08.775030148 +0000 UTC m=+1.215317767 container create 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS) 2024-09-10T18:40:09.133 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: Saving service prometheus spec with placement smithi117=a;count:1 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:09.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: Saving service prometheus spec with placement smithi117=a;count:1 2024-09-10T18:40:09.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:08 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: from='client.24442 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:09.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: Saving service prometheus spec with placement smithi117=a;count:1 2024-09-10T18:40:09.261 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.262 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:08 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:09.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:08 smithi117 podman[46815]: 2024-09-10 18:40:08.960638792 +0000 UTC m=+1.400926415 container init 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:40:09.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:08 smithi117 podman[46815]: 2024-09-10 18:40:08.964281955 +0000 UTC m=+1.404569578 container start 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T18:40:09.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 bash[46815]: 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 2024-09-10T18:40:09.262 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 systemd[1]: Started Ceph iscsi.iscsi.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:40:09.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Started the configuration object watcher 2024-09-10T18:40:09.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Checking for config object changes every 1s 2024-09-10T18:40:09.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Processing osd blocklist entries for this node 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Configuration does not have an entry for this host(smithi117.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: * Environment: production 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:40:09.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: Use a production WSGI server instead. 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: * Debug mode: off 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug * Running on all addresses. 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: * Running on all addresses. 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug * Running on http://[fe80::ec4:7aff:fe8f:4463]:5000/ (Press CTRL+C to quit) 2024-09-10T18:40:09.817 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: * Running on http://[fe80::ec4:7aff:fe8f:4463]:5000/ (Press CTRL+C to quit) 2024-09-10T18:40:10.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 178 op/s 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:10.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/4067951426' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:40:10.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 178 op/s 2024-09-10T18:40:10.232 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.232 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:10.233 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.234 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.234 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.234 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.234 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.234 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:09 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/4067951426' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: pgmap v207: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 75 KiB/s rd, 5.8 KiB/s wr, 178 op/s 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.251 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:10.252 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:09 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/4067951426' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:40:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:40:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: Adding iSCSI gateway http://:@172.21.15.117:5000 to Dashboard 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: Adding iSCSI gateway http://:@172.21.15.117:5000 to Dashboard 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:11.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:11.230 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:10 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-list"}]: dispatch 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: Adding iSCSI gateway http://:@172.21.15.117:5000 to Dashboard 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-iscsi-api-ssl-verification", "value": "true"}]: dispatch 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard iscsi-gateway-add", "name": "smithi117"}]: dispatch 2024-09-10T18:40:11.258 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:10 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:11.522 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled node-exporter update... 2024-09-10T18:40:12.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:11 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:12.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: pgmap v208: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 60 KiB/s rd, 4.6 KiB/s wr, 143 op/s 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:12.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:11 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:12.599 DEBUG:teuthology.orchestra.run.smithi066:node-exporter.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:40:12.602 DEBUG:teuthology.orchestra.run.smithi117:node-exporter.b> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:40:12.604 INFO:tasks.cephadm:Adding alertmanager.a on smithi066 2024-09-10T18:40:12.604 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi066=a' 2024-09-10T18:40:13.160 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:12 smithi117 ceph-mon[27457]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi066=a;smithi117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:13.160 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:12 smithi117 ceph-mon[27457]: Saving service node-exporter spec with placement smithi066=a;smithi117=b;count:2 2024-09-10T18:40:13.160 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:12 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:13.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[25398]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi066=a;smithi117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:13.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[25398]: Saving service node-exporter spec with placement smithi066=a;smithi117=b;count:2 2024-09-10T18:40:13.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:13.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[31695]: from='client.24478 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi066=a;smithi117=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:13.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[31695]: Saving service node-exporter spec with placement smithi066=a;smithi117=b;count:2 2024-09-10T18:40:13.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:12 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:13.556 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:14.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:13 smithi117 ceph-mon[27457]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-10T18:40:14.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:13 smithi066 ceph-mon[25398]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-10T18:40:14.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:13 smithi066 ceph-mon[31695]: pgmap v209: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 51 KiB/s rd, 3.8 KiB/s wr, 121 op/s 2024-09-10T18:40:15.298 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:15 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:15.298 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:15 smithi117 ceph-mon[27457]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.7 KiB/s wr, 46 op/s 2024-09-10T18:40:15.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:15 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:15.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:15 smithi066 ceph-mon[25398]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.7 KiB/s wr, 46 op/s 2024-09-10T18:40:15.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:15 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:15.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:15 smithi066 ceph-mon[31695]: pgmap v210: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 1.7 KiB/s wr, 46 op/s 2024-09-10T18:40:15.805 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled alertmanager update... 2024-09-10T18:40:16.554 DEBUG:teuthology.orchestra.run.smithi066:alertmanager.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:40:16.557 INFO:tasks.cephadm:Adding grafana.a on smithi117 2024-09-10T18:40:16.557 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi117=a' 2024-09-10T18:40:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi066=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: Saving service alertmanager spec with placement smithi066=a;count:1 2024-09-10T18:40:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:16.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:16.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:16.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:16 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:16.581 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.581 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi066=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:16.581 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: Saving service alertmanager spec with placement smithi066=a;count:1 2024-09-10T18:40:16.581 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.581 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='client.24484 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi066=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: Saving service alertmanager spec with placement smithi066=a;count:1 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.582 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:16.583 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:16.583 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:16.583 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:16.583 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:16 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:40:16.915 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[31695]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.9 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:18.148 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:18 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.9 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:18.149 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:17 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: pgmap v211: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.9 KiB/s rd, 921 B/s wr, 22 op/s 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:18.256 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:18.257 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:18.257 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:18.257 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:17 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:19.095 INFO:teuthology.orchestra.run.smithi117.stdout:Scheduled grafana update... 2024-09-10T18:40:19.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[25398]: Deploying daemon prometheus.a on smithi117 2024-09-10T18:40:19.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-10T18:40:19.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[31695]: Deploying daemon prometheus.a on smithi117 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-10T18:40:19.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:19 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.291 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-mon[27457]: Deploying daemon prometheus.a on smithi117 2024-09-10T18:40:19.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]: dispatch 2024-09-10T18:40:19.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]: dispatch 2024-09-10T18:40:19.292 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]: dispatch 2024-09-10T18:40:19.565 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:40:20.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.3 KiB/s rd, 820 B/s wr, 20 op/s 2024-09-10T18:40:20.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-10T18:40:20.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T18:40:20.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-10T18:40:20.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.3 KiB/s rd, 820 B/s wr, 20 op/s 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-10T18:40:20.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-10T18:40:20.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:20.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.230 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:20 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: pgmap v212: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.3 KiB/s rd, 820 B/s wr, 20 op/s 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [7, 4]}]': finished 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.a", "id": [1, 2]}]': finished 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.d", "id": [1, 5]}]': finished 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.10", "id": [1, 2]}]': finished 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.317 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:20 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:20.467 DEBUG:teuthology.orchestra.run.smithi117:grafana.a> sudo journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:40:20.510 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:40:20.510 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:40:20.767 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:21.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[31695]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:21.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[31695]: Saving service grafana spec with placement smithi117=a;count:1 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[31695]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[25398]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[25398]: Saving service grafana spec with placement smithi117=a;count:1 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:21.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:21 smithi066 ceph-mon[25398]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:21.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:21 smithi117 ceph-mon[27457]: from='client.24490 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:40:21.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:21 smithi117 ceph-mon[27457]: Saving service grafana spec with placement smithi117=a;count:1 2024-09-10T18:40:21.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:21 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:21.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:21 smithi117 ceph-mon[27457]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:40:22.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:22 smithi066 ceph-mon[25398]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-10T18:40:22.221 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:22 smithi066 ceph-mon[31695]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-10T18:40:22.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:22 smithi117 ceph-mon[27457]: pgmap v215: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 127 B/s wr, 2 op/s 2024-09-10T18:40:22.828 INFO:teuthology.orchestra.run.smithi066.stdout:[client.0] 2024-09-10T18:40:22.829 INFO:teuthology.orchestra.run.smithi066.stdout: key = AQCWkuBmQjYmMRAAzRiqWanEKDByYTw6ja+G6g== 2024-09-10T18:40:23.153 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:23 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:23.153 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:23 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:23.153 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:23 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:23.153 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:23 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:23.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:23 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:23.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:23 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/721736809' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:23.575 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:40:23.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:40:23.575 DEBUG:teuthology.orchestra.run.smithi066:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:40:23.614 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:40:24.071 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:40:24.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:24 smithi117 ceph-mon[27457]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:40:24.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:24 smithi066 ceph-mon[25398]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:40:24.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:24 smithi066 ceph-mon[31695]: pgmap v216: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 0 B/s, 0 objects/s recovering 2024-09-10T18:40:26.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:26 smithi117 ceph-mon[27457]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:26.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:26 smithi066 ceph-mon[31695]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:26.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:26 smithi066 ceph-mon[25398]: pgmap v217: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:27.874 INFO:teuthology.orchestra.run.smithi117.stdout:[client.1] 2024-09-10T18:40:27.874 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQCbkuBmuqnLMxAAz5qPbKwbhC06R2X4T+A1Tw== 2024-09-10T18:40:28.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:28 smithi117 ceph-mon[27457]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:28 smithi117 ceph-mon[27457]: from='client.? 172.21.15.117:0/3377114234' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:28 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:28 smithi117 ceph-mon[27457]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[25398]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[25398]: from='client.? 172.21.15.117:0/3377114234' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[25398]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[31695]: pgmap v218: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s; 27 B/s, 2 objects/s recovering 2024-09-10T18:40:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[31695]: from='client.? 172.21.15.117:0/3377114234' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:40:28.479 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:28 smithi066 ceph-mon[31695]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:40:28.573 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:40:28.573 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:40:28.573 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:40:28.612 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:40:28.612 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:40:28.612 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:40:28.874 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:29.524 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:29 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:40:30.264 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-mon[27457]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:40:30.264 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 systemd[1]: Starting Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:40:30.326 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:30 smithi066 ceph-mon[31695]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:40:30.327 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:30 smithi066 ceph-mon[25398]: pgmap v219: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 22 B/s, 1 objects/s recovering 2024-09-10T18:40:30.519 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 podman[48974]: 2024-09-10 18:40:30.165777997 +0000 UTC m=+0.016369534 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:40:30.520 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 podman[48974]: 2024-09-10 18:40:30.291849603 +0000 UTC m=+0.142441130 container create c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:30.773 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 podman[48974]: 2024-09-10 18:40:30.745799538 +0000 UTC m=+0.596391064 container init c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:30.774 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 podman[48974]: 2024-09-10 18:40:30.748671286 +0000 UTC m=+0.599262832 container start c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:40:30.966 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.772Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.772Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.772Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi117 (none))" 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.772Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.772Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.773Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.774Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:40:31.039 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.775Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.775Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.777Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.777Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.605µs 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.777Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.777Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.777Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=18.048µs wal_replay_duration=243.06µs wbl_replay_duration=125ns total_replay_duration=278.44µs 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.778Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.778Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.778Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.794Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.648913ms db_storage=659ns remote_storage=1.049µs web_handler=333ns query_engine=517ns scrape=395.742µs scrape_sd=15.236µs notify=633ns notify_sd=1.151µs rules=15.00706ms tracing=5.776µs 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.794Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:40:31.040 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:40:30.794Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:40:31.041 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 bash[48974]: c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 2024-09-10T18:40:31.041 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:40:30 smithi117 systemd[1]: Started Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:40:31.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/38094185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:40:31.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/38094185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:40:31.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' 2024-09-10T18:40:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:40:31.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/38094185' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:40:31.718 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6800","nonce":1514950532},{"type":"v1","addr":"172.21.15.66:6801","nonce":1514950532}]},"active_addr":"172.21.15.66:6801/1514950532","active_change":"2024-09-10T18:35:18.553271+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24112,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.66:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":307038328}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":860091355}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":1012717079}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":404238586}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.66:0","nonce":3551977885}]}]} 2024-09-10T18:40:31.722 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:40:31.722 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:40:31.722 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:40:31.997 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:32.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-mon[27457]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 963 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:40:32.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-mon[27457]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:40:32.079 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-mon[27457]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T18:40:32.080 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ignoring --setuser ceph since I am not root 2024-09-10T18:40:32.080 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ignoring --setgroup ceph since I am not root 2024-09-10T18:40:32.080 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:31 smithi117 ceph-mgr[29586]: -- 172.21.15.117:0/149303771 <== mon.1 v2:172.21.15.117:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562b3eda0340 con 0x562b3edad000 2024-09-10T18:40:32.080 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.051+0000 7f12ca19eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:40:32.203 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[31695]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 963 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:40:32.203 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[31695]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[31695]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[25398]: pgmap v220: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 963 B/s rd, 0 op/s; 20 B/s, 1 objects/s recovering 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[25398]: from='mgr.14150 172.21.15.66:0/424780240' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-mon[25398]: mgrmap e17: y(active, since 5m), standbys: x 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setuser ceph since I am not root 2024-09-10T18:40:32.204 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ignoring --setgroup ceph since I am not root 2024-09-10T18:40:32.205 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:31 smithi066 ceph-mgr[25631]: -- 172.21.15.66:0/1289291127 <== mon.0 v2:172.21.15.66:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5653cac78340 con 0x5653cac84800 2024-09-10T18:40:32.205 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.073+0000 7f39f170bec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:40:32.205 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.130+0000 7f39f170bec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:32.205 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.201+0000 7f39f170bec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:40:32.453 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.108+0000 7f12ca19eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:32.454 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.176+0000 7f12ca19eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:40:32.454 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.306+0000 7f12ca19eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:32.454 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.452+0000 7f12ca19eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:40:32.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.333+0000 7f39f170bec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:32.760 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.487+0000 7f39f170bec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:40:33.033 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.718+0000 7f12ca19eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:40:33.033 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.785+0000 7f12ca19eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:40:33.033 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:32 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:32.949+0000 7f12ca19eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:40:33.056 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.757+0000 7f39f170bec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:40:33.057 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.820+0000 7f39f170bec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:40:33.057 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:32.989+0000 7f39f170bec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:40:33.722 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:40:33.723 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":70,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","created":"2024-09-10T18:34:27.706202+0000","modified":"2024-09-10T18:40:20.002713+0000","last_up_change":"2024-09-10T18:39:43.955218+0000","last_in_change":"2024-09-10T18:39:25.004567+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:37:39.052063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:39:54.712243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:39:55.142723+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:39:56.649038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:39:58.648918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:40:00.697508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"686cf549-884c-444b-a3c4-d1f69af43911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6803","nonce":4045628832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6805","nonce":4045628832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6809","nonce":4045628832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6807","nonce":4045628832}]},"public_addr":"172.21.15.66:6803/4045628832","cluster_addr":"172.21.15.66:6805/4045628832","heartbeat_back_addr":"172.21.15.66:6809/4045628832","heartbeat_front_addr":"172.21.15.66:6807/4045628832","state":["exists","up"]},{"osd":1,"uuid":"caafa159-0619-48c7-97e3-ad1aebb5ebe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6811","nonce":3509320837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6813","nonce":3509320837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6817","nonce":3509320837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6815","nonce":3509320837}]},"public_addr":"172.21.15.66:6811/3509320837","cluster_addr":"172.21.15.66:6813/3509320837","heartbeat_back_addr":"172.21.15.66:6817/3509320837","heartbeat_front_addr":"172.21.15.66:6815/3509320837","state":["exists","up"]},{"osd":2,"uuid":"340a0fb8-062d-400d-a6e1-a6038be9bc42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6819","nonce":2361686292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6821","nonce":2361686292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6825","nonce":2361686292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6823","nonce":2361686292}]},"public_addr":"172.21.15.66:6819/2361686292","cluster_addr":"172.21.15.66:6821/2361686292","heartbeat_back_addr":"172.21.15.66:6825/2361686292","heartbeat_front_addr":"172.21.15.66:6823/2361686292","state":["exists","up"]},{"osd":3,"uuid":"e62a5562-9d0e-4d44-992f-b0da61fdc4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6827","nonce":496216846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6829","nonce":496216846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6833","nonce":496216846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6831","nonce":496216846}]},"public_addr":"172.21.15.66:6827/496216846","cluster_addr":"172.21.15.66:6829/496216846","heartbeat_back_addr":"172.21.15.66:6833/496216846","heartbeat_front_addr":"172.21.15.66:6831/496216846","state":["exists","up"]},{"osd":4,"uuid":"1a94533a-e50d-4f00-a683-a14921c963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6801","nonce":5085432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6803","nonce":5085432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6807","nonce":5085432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6805","nonce":5085432}]},"public_addr":"172.21.15.117:6801/5085432","cluster_addr":"172.21.15.117:6803/5085432","heartbeat_back_addr":"172.21.15.117:6807/5085432","heartbeat_front_addr":"172.21.15.117:6805/5085432","state":["exists","up"]},{"osd":5,"uuid":"372defb5-3bca-4453-ab60-88741d9ab57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6809","nonce":1012691608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6811","nonce":1012691608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6815","nonce":1012691608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6813","nonce":1012691608}]},"public_addr":"172.21.15.117:6809/1012691608","cluster_addr":"172.21.15.117:6811/1012691608","heartbeat_back_addr":"172.21.15.117:6815/1012691608","heartbeat_front_addr":"172.21.15.117:6813/1012691608","state":["exists","up"]},{"osd":6,"uuid":"9e25289b-3df3-477a-a98e-bc6ecf4fd7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6817","nonce":1071681872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6819","nonce":1071681872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6823","nonce":1071681872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6821","nonce":1071681872}]},"public_addr":"172.21.15.117:6817/1071681872","cluster_addr":"172.21.15.117:6819/1071681872","heartbeat_back_addr":"172.21.15.117:6823/1071681872","heartbeat_front_addr":"172.21.15.117:6821/1071681872","state":["exists","up"]},{"osd":7,"uuid":"b179067d-9cd3-40b6-978c-ae59ec89af6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6825","nonce":306391576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6827","nonce":306391576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6831","nonce":306391576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6829","nonce":306391576}]},"public_addr":"172.21.15.117:6825/306391576","cluster_addr":"172.21.15.117:6827/306391576","heartbeat_back_addr":"172.21.15.117:6831/306391576","heartbeat_front_addr":"172.21.15.117:6829/306391576","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:46.756861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:10.614445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:35.901695+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:02.748776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:27.460845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:52.832490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:17.652789+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:42.007242+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":7,"to":4}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/3684972778":"2024-09-11T18:35:18.553127+0000","172.21.15.66:6800/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/3287086836":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/2654732106":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/2959308788":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/1207688396":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/115463389":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6801/80391987":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6801/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:6800/80391987":"2024-09-11T18:34:55.725213+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:40:33.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:33 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2909507275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:33.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:33 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2909507275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:33.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:33 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:33.908+0000 7f39f170bec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:40:34.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:33 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2909507275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:34.066 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:33 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:33.795+0000 7f12ca19eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:40:34.066 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:33 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:33.859+0000 7f12ca19eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:40:34.431 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:33 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:33.977+0000 7f39f170bec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:40:34.456 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:40:34.456 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:40:34.723 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:34.964 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:34 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:34.660+0000 7f12ca19eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:40:34.964 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:34 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:34.774+0000 7f12ca19eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:40:34.964 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:34 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:34.899+0000 7f12ca19eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:34.964 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:34 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:34.962+0000 7f12ca19eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:40:35.059 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:34.746+0000 7f39f170bec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:40:35.059 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:34.863+0000 7f39f170bec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:40:35.059 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:34 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:34.994+0000 7f39f170bec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:40:35.059 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.056+0000 7f39f170bec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:40:35.216 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.089+0000 7f12ca19eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:40:35.216 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.156+0000 7f12ca19eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:40:35.216 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.214+0000 7f12ca19eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:40:35.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.192+0000 7f39f170bec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:40:35.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.261+0000 7f39f170bec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:40:35.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.323+0000 7f39f170bec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:40:35.509 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.364+0000 7f12ca19eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:40:35.509 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.435+0000 7f12ca19eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:40:35.509 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.507+0000 7f12ca19eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:40:35.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.466+0000 7f39f170bec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:40:35.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.532+0000 7f39f170bec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:40:35.604 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.601+0000 7f39f170bec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:40:35.815 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.594+0000 7f12ca19eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:35.815 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.651+0000 7f12ca19eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:40:35.815 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:35 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:35.711+0000 7f12ca19eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:40:35.884 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.685+0000 7f39f170bec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:40:35.884 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.745+0000 7f39f170bec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:40:35.884 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:35 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:35.805+0000 7f39f170bec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:40:36.617 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: Standby manager daemon x restarted 2024-09-10T18:40:36.617 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: Standby manager daemon x started 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-mon[27457]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:36.492+0000 7f12ca19eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: [10/Sep/2024:18:40:36] ENGINE Bus STARTING 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: CherryPy Checker: 2024-09-10T18:40:36.618 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: The Application mounted at '' has an empty config. 2024-09-10T18:40:36.619 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: 2024-09-10T18:40:36.619 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: [10/Sep/2024:18:40:36] ENGINE Serving on http://:::9283 2024-09-10T18:40:36.619 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: [10/Sep/2024:18:40:36] ENGINE Bus STARTED 2024-09-10T18:40:36.655 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:40:36.655 INFO:teuthology.orchestra.run.smithi066.stdout:{"epoch":71,"fsid":"f248e234-6fa2-11ef-bcea-c7b262605968","created":"2024-09-10T18:34:27.706202+0000","modified":"2024-09-10T18:40:36.630156+0000","last_up_change":"2024-09-10T18:39:43.955218+0000","last_in_change":"2024-09-10T18:39:25.004567+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:37:39.052063+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:39:54.712243+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:39:55.142723+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":65,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:39:56.649038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:39:58.648918+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:40:00.697508+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"686cf549-884c-444b-a3c4-d1f69af43911","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6802","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6803","nonce":4045628832}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6804","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6805","nonce":4045628832}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6808","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6809","nonce":4045628832}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6806","nonce":4045628832},{"type":"v1","addr":"172.21.15.66:6807","nonce":4045628832}]},"public_addr":"172.21.15.66:6803/4045628832","cluster_addr":"172.21.15.66:6805/4045628832","heartbeat_back_addr":"172.21.15.66:6809/4045628832","heartbeat_front_addr":"172.21.15.66:6807/4045628832","state":["exists","up"]},{"osd":1,"uuid":"caafa159-0619-48c7-97e3-ad1aebb5ebe0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6810","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6811","nonce":3509320837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6812","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6813","nonce":3509320837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6816","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6817","nonce":3509320837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6814","nonce":3509320837},{"type":"v1","addr":"172.21.15.66:6815","nonce":3509320837}]},"public_addr":"172.21.15.66:6811/3509320837","cluster_addr":"172.21.15.66:6813/3509320837","heartbeat_back_addr":"172.21.15.66:6817/3509320837","heartbeat_front_addr":"172.21.15.66:6815/3509320837","state":["exists","up"]},{"osd":2,"uuid":"340a0fb8-062d-400d-a6e1-a6038be9bc42","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6818","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6819","nonce":2361686292}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6820","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6821","nonce":2361686292}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6824","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6825","nonce":2361686292}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6822","nonce":2361686292},{"type":"v1","addr":"172.21.15.66:6823","nonce":2361686292}]},"public_addr":"172.21.15.66:6819/2361686292","cluster_addr":"172.21.15.66:6821/2361686292","heartbeat_back_addr":"172.21.15.66:6825/2361686292","heartbeat_front_addr":"172.21.15.66:6823/2361686292","state":["exists","up"]},{"osd":3,"uuid":"e62a5562-9d0e-4d44-992f-b0da61fdc4fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6826","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6827","nonce":496216846}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6828","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6829","nonce":496216846}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6832","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6833","nonce":496216846}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.66:6830","nonce":496216846},{"type":"v1","addr":"172.21.15.66:6831","nonce":496216846}]},"public_addr":"172.21.15.66:6827/496216846","cluster_addr":"172.21.15.66:6829/496216846","heartbeat_back_addr":"172.21.15.66:6833/496216846","heartbeat_front_addr":"172.21.15.66:6831/496216846","state":["exists","up"]},{"osd":4,"uuid":"1a94533a-e50d-4f00-a683-a14921c963f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6801","nonce":5085432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6803","nonce":5085432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6807","nonce":5085432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":5085432},{"type":"v1","addr":"172.21.15.117:6805","nonce":5085432}]},"public_addr":"172.21.15.117:6801/5085432","cluster_addr":"172.21.15.117:6803/5085432","heartbeat_back_addr":"172.21.15.117:6807/5085432","heartbeat_front_addr":"172.21.15.117:6805/5085432","state":["exists","up"]},{"osd":5,"uuid":"372defb5-3bca-4453-ab60-88741d9ab57b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6809","nonce":1012691608}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6811","nonce":1012691608}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6815","nonce":1012691608}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":1012691608},{"type":"v1","addr":"172.21.15.117:6813","nonce":1012691608}]},"public_addr":"172.21.15.117:6809/1012691608","cluster_addr":"172.21.15.117:6811/1012691608","heartbeat_back_addr":"172.21.15.117:6815/1012691608","heartbeat_front_addr":"172.21.15.117:6813/1012691608","state":["exists","up"]},{"osd":6,"uuid":"9e25289b-3df3-477a-a98e-bc6ecf4fd7fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6817","nonce":1071681872}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6819","nonce":1071681872}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6823","nonce":1071681872}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":1071681872},{"type":"v1","addr":"172.21.15.117:6821","nonce":1071681872}]},"public_addr":"172.21.15.117:6817/1071681872","cluster_addr":"172.21.15.117:6819/1071681872","heartbeat_back_addr":"172.21.15.117:6823/1071681872","heartbeat_front_addr":"172.21.15.117:6821/1071681872","state":["exists","up"]},{"osd":7,"uuid":"b179067d-9cd3-40b6-978c-ae59ec89af6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6825","nonce":306391576}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6827","nonce":306391576}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6831","nonce":306391576}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":306391576},{"type":"v1","addr":"172.21.15.117:6829","nonce":306391576}]},"public_addr":"172.21.15.117:6825/306391576","cluster_addr":"172.21.15.117:6827/306391576","heartbeat_back_addr":"172.21.15.117:6831/306391576","heartbeat_front_addr":"172.21.15.117:6829/306391576","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:36:46.756861+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:10.614445+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:37:35.901695+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:02.748776+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:27.460845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:38:52.832490+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:17.652789+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:39:42.007242+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":7,"to":4}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.d","mappings":[{"from":1,"to":5}]},{"pgid":"4.10","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.66:0/404238586":"2024-09-11T18:40:36.630107+0000","172.21.15.66:0/3551977885":"2024-09-11T18:40:36.630107+0000","172.21.15.66:6801/1514950532":"2024-09-11T18:40:36.630107+0000","172.21.15.66:6800/80391987":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6801/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:6801/80391987":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/115463389":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/1207688396":"2024-09-11T18:34:55.725213+0000","172.21.15.66:6800/1514950532":"2024-09-11T18:40:36.630107+0000","172.21.15.66:0/2654732106":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/3287086836":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/860091355":"2024-09-11T18:40:36.630107+0000","172.21.15.66:6800/4288635022":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/3684972778":"2024-09-11T18:35:18.553127+0000","172.21.15.66:0/2959308788":"2024-09-11T18:34:55.725213+0000","172.21.15.66:0/1012717079":"2024-09-11T18:40:36.630107+0000","172.21.15.66:0/307038328":"2024-09-11T18:40:36.630107+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: Standby manager daemon x restarted 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: Standby manager daemon x started 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:40:36.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[31695]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:36.626+0000 7f39f170bec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:40:36] ENGINE Bus STARTING 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: CherryPy Checker: 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: The Application mounted at '' has an empty config. 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: Standby manager daemon x restarted 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: Standby manager daemon x started 2024-09-10T18:40:36.745 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:40:36.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:40:36.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:40:36.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-mon[25398]: from='mgr.? 172.21.15.117:0/2509376025' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:40:37.022 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:40:36] ENGINE Serving on http://:::9283 2024-09-10T18:40:37.023 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:36 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:40:36] ENGINE Bus STARTED 2024-09-10T18:40:37.388 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:40:37.388 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:40:37.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:40:37.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:40:37.389 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:40:37.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:40:37.390 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:40:37.391 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: Active manager daemon y restarted 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: Activating manager daemon y 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: mgrmap e19: y(active, starting, since 0.00837358s), standbys: x 2024-09-10T18:40:37.636 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:40:37.637 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1817376624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: Manager daemon y is now available 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.638 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:40:37.639 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: Active manager daemon y restarted 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: Activating manager daemon y 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: mgrmap e19: y(active, starting, since 0.00837358s), standbys: x 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:40:37.640 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1817376624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: Manager daemon y is now available 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.641 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:37.642 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: mgrmap e18: y(active, since 5m), standbys: x 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: Active manager daemon y restarted 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: Activating manager daemon y 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: mgrmap e19: y(active, starting, since 0.00837358s), standbys: x 2024-09-10T18:40:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:40:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:40:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1817376624' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: Manager daemon y is now available 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi066", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd='[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]': finished 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:40:37.819 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:40:37.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:37.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: Updating smithi066:/etc/ceph/ceph.conf 2024-09-10T18:40:37.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: Updating smithi117:/etc/ceph/ceph.conf 2024-09-10T18:40:37.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:40:37.820 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:40:38.606 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:38.642 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:38.642 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:38.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:38.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:38.744 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: mgrmap e20: y(active, since 1.01921s), standbys: x 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:38.745 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: mgrmap e20: y(active, since 1.01921s), standbys: x 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.746 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:38 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:38.874 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:38.874 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:38.893 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: mgrmap e20: y(active, since 1.01921s), standbys: x 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: Updating smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.067 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:38 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:39.125+0000 7f3990385640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:40:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: Traceback (most recent call last): 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._connections.run(self.expiration_interval) 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._run(expiration_interval) 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:39.324 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: s = self.context.wrap_socket( 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: return self.sslsocket_class._create( 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self.do_handshake() 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: self._sslobj.do_handshake() 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:39.325 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:40:39.565 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: Deploying daemon alertmanager.a on smithi066 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[31695]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:39.926 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:39.927 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: Deploying daemon alertmanager.a on smithi066 2024-09-10T18:40:39.927 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:39.927 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:39.927 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:39 smithi066 ceph-mon[25398]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:40:40.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: Updating smithi066:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:40:40.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: Updating smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:40:40.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: Deploying daemon alertmanager.a on smithi066 2024-09-10T18:40:40.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:40.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:40.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:39 smithi117 ceph-mon[27457]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-10T18:40:41.185 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: [10/Sep/2024:18:40:39] ENGINE Bus STARTING 2024-09-10T18:40:41.185 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: [10/Sep/2024:18:40:39] ENGINE Serving on https://172.21.15.66:7150 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: [10/Sep/2024:18:40:39] ENGINE Bus STARTED 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: [10/Sep/2024:18:40:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: Traceback (most recent call last): 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: self._run(expiration_interval) 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:41.186 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: s = self.context.wrap_socket( 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: return self.sslsocket_class._create( 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: self.do_handshake() 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: self._sslobj.do_handshake() 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.187 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: [10/Sep/2024:18:40:39] ENGINE Bus STARTING 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: [10/Sep/2024:18:40:39] ENGINE Serving on https://172.21.15.66:7150 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: [10/Sep/2024:18:40:39] ENGINE Bus STARTED 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: [10/Sep/2024:18:40:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout: Traceback (most recent call last): 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:40:41.188 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._run(expiration_interval) 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: s = self.context.wrap_socket( 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:41.189 INFO:journalctl@ceph.mon.a.smithi066.stdout: return self.sslsocket_class._create( 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout: self.do_handshake() 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout: self._sslobj.do_handshake() 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.190 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:40 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: [10/Sep/2024:18:40:39] ENGINE Bus STARTING 2024-09-10T18:40:41.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: [10/Sep/2024:18:40:39] ENGINE Serving on https://172.21.15.66:7150 2024-09-10T18:40:41.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: [10/Sep/2024:18:40:39] ENGINE Bus STARTED 2024-09-10T18:40:41.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: [10/Sep/2024:18:40:39] ENGINE Error in HTTPServer.serve 2024-09-10T18:40:41.316 INFO:journalctl@ceph.mon.b.smithi117.stdout: Traceback (most recent call last): 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._run(expiration_interval) 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:40:41.317 INFO:journalctl@ceph.mon.b.smithi117.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: s = self.context.wrap_socket( 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:40:41.318 INFO:journalctl@ceph.mon.b.smithi117.stdout: return self.sslsocket_class._create( 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout: self.do_handshake() 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout: self._sslobj.do_handshake() 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:40:41.319 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:41.320 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.320 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:41.320 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:40 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:42.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:41 smithi066 ceph-mon[31695]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:42.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:41 smithi066 ceph-mon[31695]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:40:42.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:41 smithi066 ceph-mon[25398]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:42.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:41 smithi066 ceph-mon[25398]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:40:42.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:41 smithi117 ceph-mon[27457]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:42.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:41 smithi117 ceph-mon[27457]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-10T18:40:44.136 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:43 smithi066 ceph-mon[31695]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:44.136 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:43 smithi066 ceph-mon[25398]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:44.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:43 smithi117 ceph-mon[27457]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:40:45.315 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:44 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:40:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:46.019 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:45 smithi066 ceph-mon[31695]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:40:46.019 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:45 smithi066 ceph-mon[25398]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:40:46.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:45 smithi117 ceph-mon[27457]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:40:47.102 INFO:teuthology.orchestra.run.smithi066.stdout:55834574892 2024-09-10T18:40:47.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:40:47.687 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:48.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:47 smithi066 ceph-mon[31695]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:40:48.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:47 smithi066 ceph-mon[25398]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:40:48.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:47 smithi117 ceph-mon[27457]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 21 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:40:49.237 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:49.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:40:50.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:49 smithi066 ceph-mon[31695]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:50.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:49 smithi066 ceph-mon[25398]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:50.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:49 smithi117 ceph-mon[27457]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:51.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:50 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:51.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:50 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:51.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:50 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:40:52.220 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:51 smithi066 ceph-mon[31695]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:52.220 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:51 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:52.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:51 smithi066 ceph-mon[25398]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:52.221 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:51 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:52.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:51 smithi117 ceph-mon[27457]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:52.315 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:51 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:40:52.984 INFO:teuthology.orchestra.run.smithi066.stdout:34359738419 2024-09-10T18:40:52.984 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:40:54.138 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:53 smithi117 ceph-mon[27457]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:54.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:53 smithi066 ceph-mon[31695]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:54.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:53 smithi066 ceph-mon[25398]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:40:54.525 INFO:teuthology.orchestra.run.smithi066.stdout:107374182434 2024-09-10T18:40:54.526 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:40:55.124 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:40:54 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:40:54] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:40:55.130 INFO:teuthology.orchestra.run.smithi066.stdout:77309411368 2024-09-10T18:40:55.131 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:40:55.440 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.440 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[31695]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.441 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:55 smithi066 ceph-mon[25398]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:55.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:55 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:55 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:55 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:40:55.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:55 smithi117 ceph-mon[27457]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:40:56.423 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:40:56.437 INFO:teuthology.orchestra.run.smithi066.stdout:231928234001 2024-09-10T18:40:56.437 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:40:57.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:57 smithi066 ceph-mon[31695]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:57.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:57 smithi066 ceph-mon[25398]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:58.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:57 smithi117 ceph-mon[27457]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:40:58.696 INFO:teuthology.orchestra.run.smithi066.stdout:197568495636 2024-09-10T18:40:58.697 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:40:59.706 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:40:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:40:59.877 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:40:59 smithi066 ceph-mon[31695]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:40:59.877 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:40:59 smithi066 ceph-mon[25398]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:41:00.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:40:59 smithi117 ceph-mon[27457]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:41:00.844 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:00.844 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:00.845 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:00 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:01.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:00 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:01.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:00 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:01.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:00 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:01.069 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:00 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:01.563 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:01.956 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:01 smithi066 ceph-mon[31695]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:01.956 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:01 smithi066 ceph-mon[25398]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:01.967 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:01 smithi117 ceph-mon[27457]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:02.715 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:02.716 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:02.740 INFO:teuthology.orchestra.run.smithi066.stdout:163208757275 2024-09-10T18:41:02.741 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:41:03.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:03 smithi066 ceph-mon[31695]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:03.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:03 smithi066 ceph-mon[25398]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:04.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:03 smithi117 ceph-mon[27457]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:04.796 INFO:teuthology.orchestra.run.smithi066.stdout:128849018912 2024-09-10T18:41:04.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:41:05.316 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:41:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:41:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:05.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:05 smithi066 ceph-mon[25398]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:05.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:05 smithi066 ceph-mon[31695]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:06.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:05 smithi117 ceph-mon[27457]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:06.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:06 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:06.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:06 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:07.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:06 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:07.870 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:07 smithi066 ceph-mon[25398]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:07.870 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:07 smithi066 ceph-mon[31695]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:08.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:07 smithi117 ceph-mon[27457]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:09.367 INFO:teuthology.orchestra.run.smithi066.stdout:107374182438 2024-09-10T18:41:09.584 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:09.725 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE Bus STOPPING 2024-09-10T18:41:09.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/4224737100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:41:09.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.920 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:09.921 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:09 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.963 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182438 for osd.3 2024-09-10T18:41:09.963 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE Bus STOPPED 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE Bus STARTING 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/4224737100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/4224737100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:09.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:09.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:10.209 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:10.209 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:10.262 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE Serving on http://:::9283 2024-09-10T18:41:10.262 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:09 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:09] ENGINE Bus STARTED 2024-09-10T18:41:10.355 INFO:teuthology.orchestra.run.smithi066.stdout:77309411372 2024-09-10T18:41:10.846 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:10.846 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:10.846 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[31695]: Deploying daemon grafana.a on smithi117 2024-09-10T18:41:10.846 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1139761645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:41:10.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:10.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:10.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[25398]: Deploying daemon grafana.a on smithi117 2024-09-10T18:41:10.847 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:10 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1139761645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:41:11.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:10 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:11.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:10 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:41:11.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:10 smithi117 ceph-mon[27457]: Deploying daemon grafana.a on smithi117 2024-09-10T18:41:11.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:10 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1139761645' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:41:11.102 INFO:teuthology.orchestra.run.smithi066.stdout:55834574897 2024-09-10T18:41:11.937 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[31695]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:11.937 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/1528093325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:41:11.937 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2898596357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:41:11.937 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:11.937 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[25398]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:11.938 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/1528093325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:41:11.938 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2898596357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:41:11.938 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:11 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:12.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:11 smithi117 ceph-mon[27457]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:11 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/1528093325' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:41:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:11 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2898596357' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:41:12.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:11 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:12.363 INFO:teuthology.orchestra.run.smithi066.stdout:34359738423 2024-09-10T18:41:12.614 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574897 for osd.1 2024-09-10T18:41:12.614 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:13.757 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411368 got 77309411372 for osd.2 2024-09-10T18:41:13.757 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:13.966 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:13 smithi066 ceph-mon[31695]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:13.967 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:13 smithi066 ceph-mon[25398]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:14.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:13 smithi117 ceph-mon[27457]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:14.354 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738419 got 34359738423 for osd.0 2024-09-10T18:41:14.354 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:14.380 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:15.292 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:41:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:41:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:15.317 INFO:teuthology.orchestra.run.smithi066.stdout:197568495640 2024-09-10T18:41:15.525 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.525 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.525 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.525 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[31695]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:15.526 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.526 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.526 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.526 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:15 smithi066 ceph-mon[25398]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:15.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:15 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:15 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:15 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:15.566 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:15 smithi117 ceph-mon[27457]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:16.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:16 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/3867080430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:41:16.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:16 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/3867080430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:41:16.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:16 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/3867080430' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:41:17.478 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:17 smithi066 ceph-mon[31695]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:17.478 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:17 smithi066 ceph-mon[25398]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:17.565 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:17 smithi117 ceph-mon[27457]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:18.516 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495636 got 197568495640 for osd.6 2024-09-10T18:41:18.516 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:19.463 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:19.552 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:19.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:19 smithi117 ceph-mon[27457]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:19.889 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:19 smithi066 ceph-mon[31695]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:19.889 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:19 smithi066 ceph-mon[25398]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:20.765 INFO:teuthology.orchestra.run.smithi066.stdout:231928234006 2024-09-10T18:41:20.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:20 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:20.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:20 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:20.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:20 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:20.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:20 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:20.981 INFO:teuthology.orchestra.run.smithi066.stdout:163208757279 2024-09-10T18:41:21.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:20 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:21.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:20 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:21.417 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234001 got 231928234006 for osd.7 2024-09-10T18:41:21.417 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:21.712 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:21 smithi117 systemd[1]: Starting Ceph grafana.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:21.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[31695]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:21.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2600694103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:41:21.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2981616872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:41:21.944 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:21.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[25398]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:21.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2600694103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:41:21.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2981616872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:41:21.945 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:21 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:21.946 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757275 got 163208757279 for osd.5 2024-09-10T18:41:21.946 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:22.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:21 smithi117 ceph-mon[27457]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:22.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:21 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2600694103' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:41:22.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:21 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2981616872' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:41:22.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:21 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:22.066 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:21 smithi117 podman[51154]: 2024-09-10 18:41:21.651842366 +0000 UTC m=+0.014745617 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-10T18:41:22.817 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:22 smithi117 podman[51154]: 2024-09-10 18:41:22.367950421 +0000 UTC m=+0.730853665 container create d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, io.openshift.tags=base rhel8, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, release=239.1651231664, name=ubi8, description=Ceph Grafana Container, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , distribution-scope=public, architecture=x86_64, vcs-type=git) 2024-09-10T18:41:23.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:23 smithi066 ceph-mon[31695]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:23.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:23 smithi066 ceph-mon[25398]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:24.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:23 smithi117 ceph-mon[27457]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:24.409 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 podman[51154]: 2024-09-10 18:41:24.2353576 +0000 UTC m=+2.598260855 container init d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, description=Ceph Grafana Container, architecture=x86_64, name=ubi8, maintainer=Paul Cuzner , io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.openshift.expose-services=, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vendor=Red Hat, Inc., summary=Grafana Container configured for Ceph mgr/dashboard integration, io.openshift.tags=base rhel8, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-09-10T18:41:24.409 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 podman[51154]: 2024-09-10 18:41:24.238768387 +0000 UTC m=+2.601671629 container start d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., version=8.5, name=ubi8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, description=Ceph Grafana Container, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., io.buildah.version=1.32.0, architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git) 2024-09-10T18:41:24.409 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 bash[51154]: d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 systemd[1]: Started Ceph grafana.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402043596Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402224812Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402243951Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402252977Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:41:24.410 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402259327Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402265572Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402271775Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402277833Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.40228386Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402290404Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402296306Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402302221Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402308096Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=settings t=2024-09-10T18:41:24.402313981Z level=info msg="App mode production" 2024-09-10T18:41:24.411 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=sqlstore t=2024-09-10T18:41:24.402362233Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=sqlstore t=2024-09-10T18:41:24.402377855Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.403223042Z level=info msg="Starting DB migrations" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.403311057Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.403833162Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.404276554Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.405939884Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.406417269Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.407185311Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.407676884Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.408459946Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:41:24.412 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.409237209Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:41:24.660 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.409799631Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:41:24.660 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.410712923Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.41152325Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.412398546Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.413283017Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.414132191Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.414976721Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.415850648Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.416698783Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.417564327Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.418438119Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.420245796Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.421099956Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:41:24.661 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.421959532Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:41:24.662 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.422797387Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:41:24.662 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.423651018Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:41:24.662 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.424491344Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:41:24.662 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.425367159Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:41:24.662 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.428150515Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.428554866Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.428898831Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.429267074Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.429879132Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.430270637Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.43062898Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.431008819Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.431385698Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:41:24.663 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.431763968Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.432089849Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.43239743Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.432692133Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.433072992Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.433469403Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.433843076Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.434230583Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.434580136Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.434964325Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.435533906Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.435923078Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:41:24.664 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.438688256Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.441392358Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.443966449Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.444379005Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.44519212Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.446026068Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.446831156Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.447282297Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.448069051Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.448972576Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.449375639Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.450169253Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.450602053Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:41:24.665 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.451395679Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.451885057Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.452258041Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.453086159Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.45356309Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.454041379Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.45494148Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.455744647Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.45666345Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.457516615Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.458371421Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.459217553Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:41:24.666 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.45993393Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.460486821Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.461329835Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.462190126Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.46255633Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.46341886Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.464250171Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.465126621Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.465583815Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.466476142Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.467405471Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.468295572Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.468739695Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:41:24.667 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.469589049Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.47042959Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.473272691Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.475969668Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.478643999Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.481601588Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.484310713Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.484768963Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.48752165Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.490148254Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.490627435Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.492923506Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:41:24.668 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.495783876Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.498651248Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.501468498Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.501927199Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.504856496Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.507728831Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:41:24.669 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.510435889Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.511084048Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.513916783Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.516451266Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.519092813Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.521713264Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:41:24.670 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.524426234Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.527069768Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.529914019Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.530271812Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.532951426Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.533436554Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.535615929Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.538326193Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.538790867Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.541528693Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.544083479Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.546817285Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.547683649Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:41:24.671 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.550480212Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.551269889Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.551725389Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.552155002Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.552611299Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.55296616Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.553340474Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.553623695Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.554197836Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.554752828Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.555093571Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:41:24.672 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.555654989Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.556236095Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.556646709Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.557457593Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.55830097Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.558736291Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.559167595Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.559592673Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.559878042Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.560190576Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.560735515Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.561307499Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:41:24.673 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.561639217Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.56206226Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.5625081Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.56285476Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.563271708Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.563697918Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.564280883Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.56459476Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.565040222Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.565362788Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.56565478Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.566103181Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:41:24.674 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.566553886Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.566883769Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.567201552Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.567492178Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.567785642Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.568198833Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.568483768Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.569103173Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.569457329Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.57010013Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.570696668Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.571025783Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:41:24.675 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.571462954Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.574314775Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.57699166Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.579802088Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.582506891Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.585411327Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.58816469Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.590996641Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.593785199Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.596534898Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.599288234Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.60220405Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:41:24.676 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.604968042Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.607794129Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.61070101Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.613413719Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.616092928Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.618810719Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.621491862Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.623989942Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.626584161Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.629370314Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.631928364Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.634729723Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:41:24.677 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.637414066Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.640110789Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.642753799Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.645504792Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.64818739Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.650998825Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.653783393Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.656549326Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.657434833Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.658319025Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.65918669Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.660043277Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.660917047Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:41:24.678 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.661685347Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:41:24.679 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.662521248Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:41:24.679 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.663313735Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:41:24.679 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.664164339Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:41:24.679 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.66501629Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:41:24.679 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.665873373Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:41:24.912 INFO:teuthology.orchestra.run.smithi066.stdout:128849018916 2024-09-10T18:41:24.912 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:41:24] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.66669968Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.667544304Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.668334852Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.669171701Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.670053344Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:41:24.913 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.670900062Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.671753106Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.673304386Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.674186856Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.675069029Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.675911119Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.676763967Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.677618413Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.678535048Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:41:24.914 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.679403591Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:41:24.915 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.680260306Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:41:24.915 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.681112208Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:41:24.915 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.681994893Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:41:24.916 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.682893858Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:41:24.916 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.683716559Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:41:24.916 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.684571822Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.685458549Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.686296485Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.687156388Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.690222706Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.692911273Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.695740967Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.698526968Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.701371276Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:41:24.917 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.701872389Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.704693798Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.707621759Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.710519256Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.710905359Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.713847412Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.716599692Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.71945035Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.719931759Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.722619209Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.725370514Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.7281881Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:41:24.918 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.728666551Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.731287912Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.733994745Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.736552993Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.737416383Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.740112572Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.742785006Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.745545017Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.74817353Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.751015046Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.753448636Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.756191878Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:41:24.919 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.756646499Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.759384715Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.762061116Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.764904899Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.765352351Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.76803406Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.76885074Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.769717069Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.771621099Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.772458122Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.773309245Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.774173399Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.77455853Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:41:24.920 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.775433435Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.776289089Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.777095601Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.777866817Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.77871241Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.779537535Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.780366062Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.78078529Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.781594859Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.782413412Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.78327754Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.783703181Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:41:24.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.784575992Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.785436964Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.786310979Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.787328732Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.78799518Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.78891327Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.789772576Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.790584472Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.790883891Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.791200281Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.791644056Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.792095862Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.792536434Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:41:24.922 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.792811526Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:41:24.923 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.793249606Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:41:24.923 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.793654208Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.794106794Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.794545088Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.795378877Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.795812763Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.796146573Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.796535067Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:41:24.924 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.796984256Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.797414094Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.797700147Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.798144955Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.798554973Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.799004615Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.799439652Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.800228916Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.800659591Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.801084662Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.80450648Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.807916936Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.808371237Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:41:24.925 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.808778517Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.809610688Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.810046803Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.810524615Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.810980947Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.811407199Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.811683635Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.812525605Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.813374509Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.814215288Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.814637731Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.815080549Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:41:24.926 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.815894253Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.816716395Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.817154371Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.817992221Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.81881686Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.819144831Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.819573836Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.820025408Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.8204603Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.820743856Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.821609519Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.822472739Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.823353831Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:41:24.927 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.824225777Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.825121729Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.825967172Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.826858417Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.827707523Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.828620877Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.829542759Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.83039585Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.831278918Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.832177565Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.833344826Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.834227672Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:41:24.928 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.835077195Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.835937895Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.83677733Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.837662247Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.838505189Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.839314252Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.839781048Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.839892919Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.840667077Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.843515243Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.846331893Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.846466502Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.846625278Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:41:24.929 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.848938248Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.851865527Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.854507057Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.857155197Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.857645306Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.860254131Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.862988839Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.865796615Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.866260617Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.869070737Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.873677804Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.876575289Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.876954165Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:41:24.930 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.881734178Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.886096455Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.888972539Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.889459297Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.892067653Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.892500546Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.894832782Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.897494088Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.897956949Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.900546572Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.903229601Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.905836282Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:41:24.931 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.906300856Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:41:24.932 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.909038968Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:41:24.932 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.91181536Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:41:24.932 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.914536758Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:41:24.932 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.91507735Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:41:24.932 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.917799834Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:41:25.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.920479558Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.923179906Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.923645595Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.924465291Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.925291933Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.926122751Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.926933633Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.927987402Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.928662114Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.929241903Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.929765299Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.930224974Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:41:25.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.930714744Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.931194392Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.932219229Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.933014581Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.933474442Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.93374938Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:41:25.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.934207796Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.934653246Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.93508408Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.935397322Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.935737327Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.936150015Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.936609661Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.937599926Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.938090255Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.938528926Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.938980637Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.939428738Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:41:25.319 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.940471887Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.94091461Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.941340527Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.941430525Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.942208385Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.942295038Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.943037525Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.943922405Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.944789538Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.9452627Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.946130674Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.946971816Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:41:25.320 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.947852959Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.948338662Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.949136117Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.949606336Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.951935164Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.954639854Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.957683105Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.960511842Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.963255452Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.966085927Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.967153526Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.970074784Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:41:25.321 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.972856433Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.975718466Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.976215568Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.979015248Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.981828422Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.984525553Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.9850471Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.987875089Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.990800813Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.993284457Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.993614491Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.996308182Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:41:25.322 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:24.998980103Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.001682215Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.002077864Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.004756792Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.007333786Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.010076881Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.020038146Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.022760315Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.025511559Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.029171382Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:41:25.323 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.029533357Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.032445744Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.033134554Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.035348346Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.036204639Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.036525716Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=migrator t=2024-09-10T18:41:25.037294066Z level=info msg="migrations completed" performed=485 skipped=0 duration=634.012676ms 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=sqlstore t=2024-09-10T18:41:25.037697342Z level=info msg="Created default organization" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=plugin.loader t=2024-09-10T18:41:25.093719903Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=plugin.loader t=2024-09-10T18:41:25.122872113Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=plugin.loader t=2024-09-10T18:41:25.122904689Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=secrets t=2024-09-10T18:41:25.122945449Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=query_data t=2024-09-10T18:41:25.124672885Z level=info msg="Query Service initialization" 2024-09-10T18:41:25.324 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=live.push_http t=2024-09-10T18:41:25.127018159Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:41:25.653 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.653 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.653 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.653 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: Deploying daemon node-exporter.a on smithi066 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/2472895203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: Deploying daemon node-exporter.a on smithi066 2024-09-10T18:41:25.654 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:25.655 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:25 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/2472895203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:41:25.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: Deploying daemon node-exporter.a on smithi066 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:25.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/2472895203' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:41:25.970 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018912 got 128849018916 for osd.4 2024-09-10T18:41:25.971 DEBUG:teuthology.parallel:result is None 2024-09-10T18:41:25.971 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:41:25.971 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:41:26.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=infra.usagestats.collector t=2024-09-10T18:41:25.895755091Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:41:26.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=provisioning.datasources t=2024-09-10T18:41:25.896111668Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:41:26.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=provisioning.datasources t=2024-09-10T18:41:25.928581324Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:41:26.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=provisioning.datasources t=2024-09-10T18:41:25.960470224Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:41:26.316 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=provisioning.alerting t=2024-09-10T18:41:25.98216637Z level=info msg="starting to provision alerting" 2024-09-10T18:41:26.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=provisioning.alerting t=2024-09-10T18:41:25.982185739Z level=info msg="finished to provision alerting" 2024-09-10T18:41:26.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=grafanaStorageLogger t=2024-09-10T18:41:25.982262877Z level=info msg="storage starting" 2024-09-10T18:41:26.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=ngalert.state.manager t=2024-09-10T18:41:25.982257251Z level=info msg="Warming state cache for startup" 2024-09-10T18:41:26.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=http.server t=2024-09-10T18:41:25.984024863Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:41:26.317 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=ngalert.state.manager t=2024-09-10T18:41:25.992530691Z level=info msg="State cache has been initialized" states=0 duration=10.27272ms 2024-09-10T18:41:26.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=ticker t=2024-09-10T18:41:25.992580512Z level=info msg=starting first_tick=2024-09-10T18:41:30Z 2024-09-10T18:41:26.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:41:25.992590319Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:41:26.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:25 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=sqlstore.transactions t=2024-09-10T18:41:25.993575871Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:41:26.318 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:41:26 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a[51169]: logger=sqlstore.transactions t=2024-09-10T18:41:26.003956754Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T18:41:26.454 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:26.470 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:26 smithi066 systemd[1]: Starting Ceph node-exporter.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:26.737 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:26 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:26.737 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:26 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:26.767 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:26 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:27.870 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:27 smithi066 ceph-mon[31695]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:27.871 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:27 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:27.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:27 smithi066 ceph-mon[25398]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:27.871 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:27 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:27 smithi117 ceph-mon[27457]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:28.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:27 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:28.934 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:41:28.934 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:28 smithi066 bash[58568]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:41:28.940 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T18:41:29.523 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:29 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:29.632 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:29 smithi066 bash[58568]: Getting image source signatures 2024-09-10T18:41:29.632 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:29 smithi066 bash[58568]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:41:29.632 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:29 smithi066 bash[58568]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:41:29.632 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:29 smithi066 bash[58568]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:41:29.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:29 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:29.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:29 smithi117 ceph-mon[27457]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:29.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:29 smithi117 ceph-mon[27457]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[31695]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[31695]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[25398]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:29.899 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:29 smithi066 ceph-mon[25398]: from='client.24614 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:29.926 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-10T18:41:28.657239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":986,"num_read_kb":839,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1771324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":634,"ondisk_log_size":634,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386756,"kb_used_data":7916,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541692,"statfs":{"total":767926730752,"available":765482692608,"internally_reserved":0,"allocated":8105984,"data_stored":4423980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002996"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007348+0000","last_change":"2024-09-10T18:40:02.643950+0000","last_active":"2024-09-10T18:40:20.007348+0000","last_peered":"2024-09-10T18:40:20.007348+0000","last_clean":"2024-09-10T18:40:20.007348+0000","last_became_active":"2024-09-10T18:40:02.643334+0000","last_became_peered":"2024-09-10T18:40:02.643334+0000","last_unstale":"2024-09-10T18:40:20.007348+0000","last_undegraded":"2024-09-10T18:40:20.007348+0000","last_fullsized":"2024-09-10T18:40:20.007348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.198514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243117+0000","last_change":"2024-09-10T18:39:56.176131+0000","last_active":"2024-09-10T18:40:20.243117+0000","last_peered":"2024-09-10T18:40:20.243117+0000","last_clean":"2024-09-10T18:40:20.243117+0000","last_became_active":"2024-09-10T18:39:56.175827+0000","last_became_peered":"2024-09-10T18:39:56.175827+0000","last_unstale":"2024-09-10T18:40:20.243117+0000","last_undegraded":"2024-09-10T18:40:20.243117+0000","last_fullsized":"2024-09-10T18:40:20.243117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:40.111091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007241+0000","last_change":"2024-09-10T18:39:58.184433+0000","last_active":"2024-09-10T18:40:20.007241+0000","last_peered":"2024-09-10T18:40:20.007241+0000","last_clean":"2024-09-10T18:40:20.007241+0000","last_became_active":"2024-09-10T18:39:58.184161+0000","last_became_peered":"2024-09-10T18:39:58.184161+0000","last_unstale":"2024-09-10T18:40:20.007241+0000","last_undegraded":"2024-09-10T18:40:20.007241+0000","last_fullsized":"2024-09-10T18:40:20.007241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:41.953694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008153+0000","last_change":"2024-09-10T18:40:00.204158+0000","last_active":"2024-09-10T18:40:20.008153+0000","last_peered":"2024-09-10T18:40:20.008153+0000","last_clean":"2024-09-10T18:40:20.008153+0000","last_became_active":"2024-09-10T18:40:00.203865+0000","last_became_peered":"2024-09-10T18:40:00.203865+0000","last_unstale":"2024-09-10T18:40:20.008153+0000","last_undegraded":"2024-09-10T18:40:20.008153+0000","last_fullsized":"2024-09-10T18:40:20.008153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:46.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008378+0000","last_change":"2024-09-10T18:39:56.178829+0000","last_active":"2024-09-10T18:40:20.008378+0000","last_peered":"2024-09-10T18:40:20.008378+0000","last_clean":"2024-09-10T18:40:20.008378+0000","last_became_active":"2024-09-10T18:39:56.178254+0000","last_became_peered":"2024-09-10T18:39:56.178254+0000","last_unstale":"2024-09-10T18:40:20.008378+0000","last_undegraded":"2024-09-10T18:40:20.008378+0000","last_fullsized":"2024-09-10T18:40:20.008378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:25.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008228+0000","last_change":"2024-09-10T18:39:58.188976+0000","last_active":"2024-09-10T18:40:20.008228+0000","last_peered":"2024-09-10T18:40:20.008228+0000","last_clean":"2024-09-10T18:40:20.008228+0000","last_became_active":"2024-09-10T18:39:58.188679+0000","last_became_peered":"2024-09-10T18:39:58.188679+0000","last_unstale":"2024-09-10T18:40:20.008228+0000","last_undegraded":"2024-09-10T18:40:20.008228+0000","last_fullsized":"2024-09-10T18:40:20.008228+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:37.582865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009655+0000","last_change":"2024-09-10T18:40:00.205242+0000","last_active":"2024-09-10T18:40:20.009655+0000","last_peered":"2024-09-10T18:40:20.009655+0000","last_clean":"2024-09-10T18:40:20.009655+0000","last_became_active":"2024-09-10T18:40:00.204963+0000","last_became_peered":"2024-09-10T18:40:00.204963+0000","last_unstale":"2024-09-10T18:40:20.009655+0000","last_undegraded":"2024-09-10T18:40:20.009655+0000","last_fullsized":"2024-09-10T18:40:20.009655+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:50.342424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008460+0000","last_change":"2024-09-10T18:40:02.218319+0000","last_active":"2024-09-10T18:40:20.008460+0000","last_peered":"2024-09-10T18:40:20.008460+0000","last_clean":"2024-09-10T18:40:20.008460+0000","last_became_active":"2024-09-10T18:40:02.218026+0000","last_became_peered":"2024-09-10T18:40:02.218026+0000","last_unstale":"2024-09-10T18:40:20.008460+0000","last_undegraded":"2024-09-10T18:40:20.008460+0000","last_fullsized":"2024-09-10T18:40:20.008460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:33.343417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006716+0000","last_change":"2024-09-10T18:39:56.170140+0000","last_active":"2024-09-10T18:40:20.006716+0000","last_peered":"2024-09-10T18:40:20.006716+0000","last_clean":"2024-09-10T18:40:20.006716+0000","last_became_active":"2024-09-10T18:39:56.169876+0000","last_became_peered":"2024-09-10T18:39:56.169876+0000","last_unstale":"2024-09-10T18:40:20.006716+0000","last_undegraded":"2024-09-10T18:40:20.006716+0000","last_fullsized":"2024-09-10T18:40:20.006716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:15.453448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009099+0000","last_change":"2024-09-10T18:39:58.189175+0000","last_active":"2024-09-10T18:40:20.009099+0000","last_peered":"2024-09-10T18:40:20.009099+0000","last_clean":"2024-09-10T18:40:20.009099+0000","last_became_active":"2024-09-10T18:39:58.188879+0000","last_became_peered":"2024-09-10T18:39:58.188879+0000","last_unstale":"2024-09-10T18:40:20.009099+0000","last_undegraded":"2024-09-10T18:40:20.009099+0000","last_fullsized":"2024-09-10T18:40:20.009099+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:38.697748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006161+0000","last_change":"2024-09-10T18:40:00.199160+0000","last_active":"2024-09-10T18:40:20.006161+0000","last_peered":"2024-09-10T18:40:20.006161+0000","last_clean":"2024-09-10T18:40:20.006161+0000","last_became_active":"2024-09-10T18:40:00.198905+0000","last_became_peered":"2024-09-10T18:40:00.198905+0000","last_unstale":"2024-09-10T18:40:20.006161+0000","last_undegraded":"2024-09-10T18:40:20.006161+0000","last_fullsized":"2024-09-10T18:40:20.006161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:20.323503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244106+0000","last_change":"2024-09-10T18:40:02.219005+0000","last_active":"2024-09-10T18:40:20.244106+0000","last_peered":"2024-09-10T18:40:20.244106+0000","last_clean":"2024-09-10T18:40:20.244106+0000","last_became_active":"2024-09-10T18:40:02.218275+0000","last_became_peered":"2024-09-10T18:40:02.218275+0000","last_unstale":"2024-09-10T18:40:20.244106+0000","last_undegraded":"2024-09-10T18:40:20.244106+0000","last_fullsized":"2024-09-10T18:40:20.244106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:57.728322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007209+0000","last_change":"2024-09-10T18:39:56.170759+0000","last_active":"2024-09-10T18:40:20.007209+0000","last_peered":"2024-09-10T18:40:20.007209+0000","last_clean":"2024-09-10T18:40:20.007209+0000","last_became_active":"2024-09-10T18:39:56.170498+0000","last_became_peered":"2024-09-10T18:39:56.170498+0000","last_unstale":"2024-09-10T18:40:20.007209+0000","last_undegraded":"2024-09-10T18:40:20.007209+0000","last_fullsized":"2024-09-10T18:40:20.007209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:32.416607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008706+0000","last_change":"2024-09-10T18:39:58.186155+0000","last_active":"2024-09-10T18:40:20.008706+0000","last_peered":"2024-09-10T18:40:20.008706+0000","last_clean":"2024-09-10T18:40:20.008706+0000","last_became_active":"2024-09-10T18:39:58.185409+0000","last_became_peered":"2024-09-10T18:39:58.185409+0000","last_unstale":"2024-09-10T18:40:20.008706+0000","last_undegraded":"2024-09-10T18:40:20.008706+0000","last_fullsized":"2024-09-10T18:40:20.008706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:46.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244142+0000","last_change":"2024-09-10T18:40:00.200041+0000","last_active":"2024-09-10T18:40:20.244142+0000","last_peered":"2024-09-10T18:40:20.244142+0000","last_clean":"2024-09-10T18:40:20.244142+0000","last_became_active":"2024-09-10T18:40:00.199758+0000","last_became_peered":"2024-09-10T18:40:00.199758+0000","last_unstale":"2024-09-10T18:40:20.244142+0000","last_undegraded":"2024-09-10T18:40:20.244142+0000","last_fullsized":"2024-09-10T18:40:20.244142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:24.375347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243497+0000","last_change":"2024-09-10T18:40:02.218941+0000","last_active":"2024-09-10T18:40:20.243497+0000","last_peered":"2024-09-10T18:40:20.243497+0000","last_clean":"2024-09-10T18:40:20.243497+0000","last_became_active":"2024-09-10T18:40:02.218680+0000","last_became_peered":"2024-09-10T18:40:02.218680+0000","last_unstale":"2024-09-10T18:40:20.243497+0000","last_undegraded":"2024-09-10T18:40:20.243497+0000","last_fullsized":"2024-09-10T18:40:20.243497+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:49.826784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":74,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.254933+0000","last_change":"2024-09-10T18:40:20.254564+0000","last_active":"2024-09-10T18:40:20.254933+0000","last_peered":"2024-09-10T18:40:20.254933+0000","last_clean":"2024-09-10T18:40:20.254933+0000","last_became_active":"2024-09-10T18:40:20.245552+0000","last_became_peered":"2024-09-10T18:40:20.245552+0000","last_unstale":"2024-09-10T18:40:20.254933+0000","last_undegraded":"2024-09-10T18:40:20.254933+0000","last_fullsized":"2024-09-10T18:40:20.254933+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:51.213000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006709+0000","last_change":"2024-09-10T18:39:56.168767+0000","last_active":"2024-09-10T18:40:20.006709+0000","last_peered":"2024-09-10T18:40:20.006709+0000","last_clean":"2024-09-10T18:40:20.006709+0000","last_became_active":"2024-09-10T18:39:56.168515+0000","last_became_peered":"2024-09-10T18:39:56.168515+0000","last_unstale":"2024-09-10T18:40:20.006709+0000","last_undegraded":"2024-09-10T18:40:20.006709+0000","last_fullsized":"2024-09-10T18:40:20.006709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:50.679751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011957+0000","last_change":"2024-09-10T18:40:00.199924+0000","last_active":"2024-09-10T18:40:20.011957+0000","last_peered":"2024-09-10T18:40:20.011957+0000","last_clean":"2024-09-10T18:40:20.011957+0000","last_became_active":"2024-09-10T18:40:00.199629+0000","last_became_peered":"2024-09-10T18:40:00.199629+0000","last_unstale":"2024-09-10T18:40:20.011957+0000","last_undegraded":"2024-09-10T18:40:20.011957+0000","last_fullsized":"2024-09-10T18:40:20.011957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:36.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006193+0000","last_change":"2024-09-10T18:40:02.218791+0000","last_active":"2024-09-10T18:40:20.006193+0000","last_peered":"2024-09-10T18:40:20.006193+0000","last_clean":"2024-09-10T18:40:20.006193+0000","last_became_active":"2024-09-10T18:40:02.218530+0000","last_became_peered":"2024-09-10T18:40:02.218530+0000","last_unstale":"2024-09-10T18:40:20.006193+0000","last_undegraded":"2024-09-10T18:40:20.006193+0000","last_fullsized":"2024-09-10T18:40:20.006193+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:33.515989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009270+0000","last_change":"2024-09-10T18:39:58.188803+0000","last_active":"2024-09-10T18:40:20.009270+0000","last_peered":"2024-09-10T18:40:20.009270+0000","last_clean":"2024-09-10T18:40:20.009270+0000","last_became_active":"2024-09-10T18:39:58.188075+0000","last_became_peered":"2024-09-10T18:39:58.188075+0000","last_unstale":"2024-09-10T18:40:20.009270+0000","last_undegraded":"2024-09-10T18:40:20.009270+0000","last_fullsized":"2024-09-10T18:40:20.009270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:57.195997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011212+0000","last_change":"2024-09-10T18:39:56.175101+0000","last_active":"2024-09-10T18:40:20.011212+0000","last_peered":"2024-09-10T18:40:20.011212+0000","last_clean":"2024-09-10T18:40:20.011212+0000","last_became_active":"2024-09-10T18:39:56.173634+0000","last_became_peered":"2024-09-10T18:39:56.173634+0000","last_unstale":"2024-09-10T18:40:20.011212+0000","last_undegraded":"2024-09-10T18:40:20.011212+0000","last_fullsized":"2024-09-10T18:40:20.011212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.655949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294880+0000","last_change":"2024-09-10T18:40:00.203020+0000","last_active":"2024-09-10T18:41:19.294880+0000","last_peered":"2024-09-10T18:41:19.294880+0000","last_clean":"2024-09-10T18:41:19.294880+0000","last_became_active":"2024-09-10T18:40:00.202719+0000","last_became_peered":"2024-09-10T18:40:00.202719+0000","last_unstale":"2024-09-10T18:41:19.294880+0000","last_undegraded":"2024-09-10T18:41:19.294880+0000","last_fullsized":"2024-09-10T18:41:19.294880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:03.397133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008706+0000","last_change":"2024-09-10T18:40:02.213562+0000","last_active":"2024-09-10T18:40:20.008706+0000","last_peered":"2024-09-10T18:40:20.008706+0000","last_clean":"2024-09-10T18:40:20.008706+0000","last_became_active":"2024-09-10T18:40:02.213308+0000","last_became_peered":"2024-09-10T18:40:02.213308+0000","last_unstale":"2024-09-10T18:40:20.008706+0000","last_undegraded":"2024-09-10T18:40:20.008706+0000","last_fullsized":"2024-09-10T18:40:20.008706+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011858+0000","last_change":"2024-09-10T18:39:58.189590+0000","last_active":"2024-09-10T18:40:20.011858+0000","last_peered":"2024-09-10T18:40:20.011858+0000","last_clean":"2024-09-10T18:40:20.011858+0000","last_became_active":"2024-09-10T18:39:58.189261+0000","last_became_peered":"2024-09-10T18:39:58.189261+0000","last_unstale":"2024-09-10T18:40:20.011858+0000","last_undegraded":"2024-09-10T18:40:20.011858+0000","last_fullsized":"2024-09-10T18:40:20.011858+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:00.793497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010524+0000","last_change":"2024-09-10T18:39:56.176475+0000","last_active":"2024-09-10T18:40:20.010524+0000","last_peered":"2024-09-10T18:40:20.010524+0000","last_clean":"2024-09-10T18:40:20.010524+0000","last_became_active":"2024-09-10T18:39:56.173821+0000","last_became_peered":"2024-09-10T18:39:56.173821+0000","last_unstale":"2024-09-10T18:40:20.010524+0000","last_undegraded":"2024-09-10T18:40:20.010524+0000","last_fullsized":"2024-09-10T18:40:20.010524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:54.059952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295073+0000","last_change":"2024-09-10T18:40:00.205245+0000","last_active":"2024-09-10T18:41:19.295073+0000","last_peered":"2024-09-10T18:41:19.295073+0000","last_clean":"2024-09-10T18:41:19.295073+0000","last_became_active":"2024-09-10T18:40:00.204887+0000","last_became_peered":"2024-09-10T18:40:00.204887+0000","last_unstale":"2024-09-10T18:41:19.295073+0000","last_undegraded":"2024-09-10T18:41:19.295073+0000","last_fullsized":"2024-09-10T18:41:19.295073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:34.589332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244801+0000","last_change":"2024-09-10T18:40:02.217599+0000","last_active":"2024-09-10T18:40:20.244801+0000","last_peered":"2024-09-10T18:40:20.244801+0000","last_clean":"2024-09-10T18:40:20.244801+0000","last_became_active":"2024-09-10T18:40:02.217314+0000","last_became_peered":"2024-09-10T18:40:02.217314+0000","last_unstale":"2024-09-10T18:40:20.244801+0000","last_undegraded":"2024-09-10T18:40:20.244801+0000","last_fullsized":"2024-09-10T18:40:20.244801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:37.108476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008990+0000","last_change":"2024-09-10T18:39:58.188567+0000","last_active":"2024-09-10T18:40:20.008990+0000","last_peered":"2024-09-10T18:40:20.008990+0000","last_clean":"2024-09-10T18:40:20.008990+0000","last_became_active":"2024-09-10T18:39:58.188189+0000","last_became_peered":"2024-09-10T18:39:58.188189+0000","last_unstale":"2024-09-10T18:40:20.008990+0000","last_undegraded":"2024-09-10T18:40:20.008990+0000","last_fullsized":"2024-09-10T18:40:20.008990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:37.758602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006154+0000","last_change":"2024-09-10T18:39:56.171415+0000","last_active":"2024-09-10T18:40:20.006154+0000","last_peered":"2024-09-10T18:40:20.006154+0000","last_clean":"2024-09-10T18:40:20.006154+0000","last_became_active":"2024-09-10T18:39:56.170065+0000","last_became_peered":"2024-09-10T18:39:56.170065+0000","last_unstale":"2024-09-10T18:40:20.006154+0000","last_undegraded":"2024-09-10T18:40:20.006154+0000","last_fullsized":"2024-09-10T18:40:20.006154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:54.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244481+0000","last_change":"2024-09-10T18:40:00.206942+0000","last_active":"2024-09-10T18:40:20.244481+0000","last_peered":"2024-09-10T18:40:20.244481+0000","last_clean":"2024-09-10T18:40:20.244481+0000","last_became_active":"2024-09-10T18:40:00.206630+0000","last_became_peered":"2024-09-10T18:40:00.206630+0000","last_unstale":"2024-09-10T18:40:20.244481+0000","last_undegraded":"2024-09-10T18:40:20.244481+0000","last_fullsized":"2024-09-10T18:40:20.244481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:12.397846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007438+0000","last_change":"2024-09-10T18:40:02.643603+0000","last_active":"2024-09-10T18:40:20.007438+0000","last_peered":"2024-09-10T18:40:20.007438+0000","last_clean":"2024-09-10T18:40:20.007438+0000","last_became_active":"2024-09-10T18:40:02.643128+0000","last_became_peered":"2024-09-10T18:40:02.643128+0000","last_unstale":"2024-09-10T18:40:20.007438+0000","last_undegraded":"2024-09-10T18:40:20.007438+0000","last_fullsized":"2024-09-10T18:40:20.007438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:49.775594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008652+0000","last_change":"2024-09-10T18:39:58.187128+0000","last_active":"2024-09-10T18:40:20.008652+0000","last_peered":"2024-09-10T18:40:20.008652+0000","last_clean":"2024-09-10T18:40:20.008652+0000","last_became_active":"2024-09-10T18:39:58.186677+0000","last_became_peered":"2024-09-10T18:39:58.186677+0000","last_unstale":"2024-09-10T18:40:20.008652+0000","last_undegraded":"2024-09-10T18:40:20.008652+0000","last_fullsized":"2024-09-10T18:40:20.008652+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:26.652612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008695+0000","last_change":"2024-09-10T18:39:56.168526+0000","last_active":"2024-09-10T18:40:20.008695+0000","last_peered":"2024-09-10T18:40:20.008695+0000","last_clean":"2024-09-10T18:40:20.008695+0000","last_became_active":"2024-09-10T18:39:56.168258+0000","last_became_peered":"2024-09-10T18:39:56.168258+0000","last_unstale":"2024-09-10T18:40:20.008695+0000","last_undegraded":"2024-09-10T18:40:20.008695+0000","last_fullsized":"2024-09-10T18:40:20.008695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:29.952461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294888+0000","last_change":"2024-09-10T18:40:00.202205+0000","last_active":"2024-09-10T18:41:19.294888+0000","last_peered":"2024-09-10T18:41:19.294888+0000","last_clean":"2024-09-10T18:41:19.294888+0000","last_became_active":"2024-09-10T18:40:00.201897+0000","last_became_peered":"2024-09-10T18:40:00.201897+0000","last_unstale":"2024-09-10T18:41:19.294888+0000","last_undegraded":"2024-09-10T18:41:19.294888+0000","last_fullsized":"2024-09-10T18:41:19.294888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:10.127581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006546+0000","last_change":"2024-09-10T18:40:02.645133+0000","last_active":"2024-09-10T18:40:20.006546+0000","last_peered":"2024-09-10T18:40:20.006546+0000","last_clean":"2024-09-10T18:40:20.006546+0000","last_became_active":"2024-09-10T18:40:02.644840+0000","last_became_peered":"2024-09-10T18:40:02.644840+0000","last_unstale":"2024-09-10T18:40:20.006546+0000","last_undegraded":"2024-09-10T18:40:20.006546+0000","last_fullsized":"2024-09-10T18:40:20.006546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:45.748017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006996+0000","last_change":"2024-09-10T18:39:58.185421+0000","last_active":"2024-09-10T18:40:20.006996+0000","last_peered":"2024-09-10T18:40:20.006996+0000","last_clean":"2024-09-10T18:40:20.006996+0000","last_became_active":"2024-09-10T18:39:58.184795+0000","last_became_peered":"2024-09-10T18:39:58.184795+0000","last_unstale":"2024-09-10T18:40:20.006996+0000","last_undegraded":"2024-09-10T18:40:20.006996+0000","last_fullsized":"2024-09-10T18:40:20.006996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.224265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"60'1","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011583+0000","last_change":"2024-09-10T18:39:56.170415+0000","last_active":"2024-09-10T18:40:20.011583+0000","last_peered":"2024-09-10T18:40:20.011583+0000","last_clean":"2024-09-10T18:40:20.011583+0000","last_became_active":"2024-09-10T18:39:56.170127+0000","last_became_peered":"2024-09-10T18:39:56.170127+0000","last_unstale":"2024-09-10T18:40:20.011583+0000","last_undegraded":"2024-09-10T18:40:20.011583+0000","last_fullsized":"2024-09-10T18:40:20.011583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:45.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008499+0000","last_change":"2024-09-10T18:40:00.198481+0000","last_active":"2024-09-10T18:40:20.008499+0000","last_peered":"2024-09-10T18:40:20.008499+0000","last_clean":"2024-09-10T18:40:20.008499+0000","last_became_active":"2024-09-10T18:40:00.198066+0000","last_became_peered":"2024-09-10T18:40:00.198066+0000","last_unstale":"2024-09-10T18:40:20.008499+0000","last_undegraded":"2024-09-10T18:40:20.008499+0000","last_fullsized":"2024-09-10T18:40:20.008499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:11.422250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008629+0000","last_change":"2024-09-10T18:40:02.214195+0000","last_active":"2024-09-10T18:40:20.008629+0000","last_peered":"2024-09-10T18:40:20.008629+0000","last_clean":"2024-09-10T18:40:20.008629+0000","last_became_active":"2024-09-10T18:40:02.213936+0000","last_became_peered":"2024-09-10T18:40:02.213936+0000","last_unstale":"2024-09-10T18:40:20.008629+0000","last_undegraded":"2024-09-10T18:40:20.008629+0000","last_fullsized":"2024-09-10T18:40:20.008629+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:06.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244141+0000","last_change":"2024-09-10T18:39:58.183826+0000","last_active":"2024-09-10T18:40:20.244141+0000","last_peered":"2024-09-10T18:40:20.244141+0000","last_clean":"2024-09-10T18:40:20.244141+0000","last_became_active":"2024-09-10T18:39:58.183542+0000","last_became_peered":"2024-09-10T18:39:58.183542+0000","last_unstale":"2024-09-10T18:40:20.244141+0000","last_undegraded":"2024-09-10T18:40:20.244141+0000","last_fullsized":"2024-09-10T18:40:20.244141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:02.366162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006760+0000","last_change":"2024-09-10T18:39:56.169355+0000","last_active":"2024-09-10T18:40:20.006760+0000","last_peered":"2024-09-10T18:40:20.006760+0000","last_clean":"2024-09-10T18:40:20.006760+0000","last_became_active":"2024-09-10T18:39:56.169102+0000","last_became_peered":"2024-09-10T18:39:56.169102+0000","last_unstale":"2024-09-10T18:40:20.006760+0000","last_undegraded":"2024-09-10T18:40:20.006760+0000","last_fullsized":"2024-09-10T18:40:20.006760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:23.264894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008348+0000","last_change":"2024-09-10T18:40:00.201031+0000","last_active":"2024-09-10T18:40:20.008348+0000","last_peered":"2024-09-10T18:40:20.008348+0000","last_clean":"2024-09-10T18:40:20.008348+0000","last_became_active":"2024-09-10T18:40:00.200703+0000","last_became_peered":"2024-09-10T18:40:00.200703+0000","last_unstale":"2024-09-10T18:40:20.008348+0000","last_undegraded":"2024-09-10T18:40:20.008348+0000","last_fullsized":"2024-09-10T18:40:20.008348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:58.060838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009546+0000","last_change":"2024-09-10T18:40:02.644996+0000","last_active":"2024-09-10T18:40:20.009546+0000","last_peered":"2024-09-10T18:40:20.009546+0000","last_clean":"2024-09-10T18:40:20.009546+0000","last_became_active":"2024-09-10T18:40:02.644619+0000","last_became_peered":"2024-09-10T18:40:02.644619+0000","last_unstale":"2024-09-10T18:40:20.009546+0000","last_undegraded":"2024-09-10T18:40:20.009546+0000","last_fullsized":"2024-09-10T18:40:20.009546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:07.209586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011077+0000","last_change":"2024-09-10T18:39:58.187849+0000","last_active":"2024-09-10T18:40:20.011077+0000","last_peered":"2024-09-10T18:40:20.011077+0000","last_clean":"2024-09-10T18:40:20.011077+0000","last_became_active":"2024-09-10T18:39:58.187225+0000","last_became_peered":"2024-09-10T18:39:58.187225+0000","last_unstale":"2024-09-10T18:40:20.011077+0000","last_undegraded":"2024-09-10T18:40:20.011077+0000","last_fullsized":"2024-09-10T18:40:20.011077+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:52.072943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.012098+0000","last_change":"2024-09-10T18:39:56.173477+0000","last_active":"2024-09-10T18:40:20.012098+0000","last_peered":"2024-09-10T18:40:20.012098+0000","last_clean":"2024-09-10T18:40:20.012098+0000","last_became_active":"2024-09-10T18:39:56.173001+0000","last_became_peered":"2024-09-10T18:39:56.173001+0000","last_unstale":"2024-09-10T18:40:20.012098+0000","last_undegraded":"2024-09-10T18:40:20.012098+0000","last_fullsized":"2024-09-10T18:40:20.012098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:46.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294880+0000","last_change":"2024-09-10T18:40:00.201429+0000","last_active":"2024-09-10T18:41:19.294880+0000","last_peered":"2024-09-10T18:41:19.294880+0000","last_clean":"2024-09-10T18:41:19.294880+0000","last_became_active":"2024-09-10T18:40:00.201114+0000","last_became_peered":"2024-09-10T18:40:00.201114+0000","last_unstale":"2024-09-10T18:41:19.294880+0000","last_undegraded":"2024-09-10T18:41:19.294880+0000","last_fullsized":"2024-09-10T18:41:19.294880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:13.232098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243319+0000","last_change":"2024-09-10T18:40:02.222334+0000","last_active":"2024-09-10T18:40:20.243319+0000","last_peered":"2024-09-10T18:40:20.243319+0000","last_clean":"2024-09-10T18:40:20.243319+0000","last_became_active":"2024-09-10T18:40:02.222044+0000","last_became_peered":"2024-09-10T18:40:02.222044+0000","last_unstale":"2024-09-10T18:40:20.243319+0000","last_undegraded":"2024-09-10T18:40:20.243319+0000","last_fullsized":"2024-09-10T18:40:20.243319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:11.906709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"67'30","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294905+0000","last_change":"2024-09-10T18:39:58.182811+0000","last_active":"2024-09-10T18:41:19.294905+0000","last_peered":"2024-09-10T18:41:19.294905+0000","last_clean":"2024-09-10T18:41:19.294905+0000","last_became_active":"2024-09-10T18:39:58.182229+0000","last_became_peered":"2024-09-10T18:39:58.182229+0000","last_unstale":"2024-09-10T18:41:19.294905+0000","last_undegraded":"2024-09-10T18:41:19.294905+0000","last_fullsized":"2024-09-10T18:41:19.294905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:59.638710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243141+0000","last_change":"2024-09-10T18:39:56.168390+0000","last_active":"2024-09-10T18:40:20.243141+0000","last_peered":"2024-09-10T18:40:20.243141+0000","last_clean":"2024-09-10T18:40:20.243141+0000","last_became_active":"2024-09-10T18:39:56.167729+0000","last_became_peered":"2024-09-10T18:39:56.167729+0000","last_unstale":"2024-09-10T18:40:20.243141+0000","last_undegraded":"2024-09-10T18:40:20.243141+0000","last_fullsized":"2024-09-10T18:40:20.243141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:05.076803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244062+0000","last_change":"2024-09-10T18:40:00.204429+0000","last_active":"2024-09-10T18:40:20.244062+0000","last_peered":"2024-09-10T18:40:20.244062+0000","last_clean":"2024-09-10T18:40:20.244062+0000","last_became_active":"2024-09-10T18:40:00.204109+0000","last_became_peered":"2024-09-10T18:40:00.204109+0000","last_unstale":"2024-09-10T18:40:20.244062+0000","last_undegraded":"2024-09-10T18:40:20.244062+0000","last_fullsized":"2024-09-10T18:40:20.244062+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.127260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007926+0000","last_change":"2024-09-10T18:40:02.215984+0000","last_active":"2024-09-10T18:40:20.007926+0000","last_peered":"2024-09-10T18:40:20.007926+0000","last_clean":"2024-09-10T18:40:20.007926+0000","last_became_active":"2024-09-10T18:40:02.215685+0000","last_became_peered":"2024-09-10T18:40:02.215685+0000","last_unstale":"2024-09-10T18:40:20.007926+0000","last_undegraded":"2024-09-10T18:40:20.007926+0000","last_fullsized":"2024-09-10T18:40:20.007926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:24.849917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":45,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011985+0000","last_change":"2024-09-10T18:39:58.183557+0000","last_active":"2024-09-10T18:40:20.011985+0000","last_peered":"2024-09-10T18:40:20.011985+0000","last_clean":"2024-09-10T18:40:20.011985+0000","last_became_active":"2024-09-10T18:39:58.183267+0000","last_became_peered":"2024-09-10T18:39:58.183267+0000","last_unstale":"2024-09-10T18:40:20.011985+0000","last_undegraded":"2024-09-10T18:40:20.011985+0000","last_fullsized":"2024-09-10T18:40:20.011985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:04.112290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":112,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:26.400278+0000","last_change":"2024-09-10T18:40:01.199339+0000","last_active":"2024-09-10T18:41:26.400278+0000","last_peered":"2024-09-10T18:41:26.400278+0000","last_clean":"2024-09-10T18:41:26.400278+0000","last_became_active":"2024-09-10T18:39:56.169389+0000","last_became_peered":"2024-09-10T18:39:56.169389+0000","last_unstale":"2024-09-10T18:41:26.400278+0000","last_undegraded":"2024-09-10T18:41:26.400278+0000","last_fullsized":"2024-09-10T18:41:26.400278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:26.751398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450086,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":84,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006871+0000","last_change":"2024-09-10T18:39:56.175630+0000","last_active":"2024-09-10T18:40:20.006871+0000","last_peered":"2024-09-10T18:40:20.006871+0000","last_clean":"2024-09-10T18:40:20.006871+0000","last_became_active":"2024-09-10T18:39:56.175331+0000","last_became_peered":"2024-09-10T18:39:56.175331+0000","last_unstale":"2024-09-10T18:40:20.006871+0000","last_undegraded":"2024-09-10T18:40:20.006871+0000","last_fullsized":"2024-09-10T18:40:20.006871+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:39.721081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006241+0000","last_change":"2024-09-10T18:40:00.198067+0000","last_active":"2024-09-10T18:40:20.006241+0000","last_peered":"2024-09-10T18:40:20.006241+0000","last_clean":"2024-09-10T18:40:20.006241+0000","last_became_active":"2024-09-10T18:40:00.197805+0000","last_became_peered":"2024-09-10T18:40:00.197805+0000","last_unstale":"2024-09-10T18:40:20.006241+0000","last_undegraded":"2024-09-10T18:40:20.006241+0000","last_fullsized":"2024-09-10T18:40:20.006241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:20.013355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006508+0000","last_change":"2024-09-10T18:40:02.643088+0000","last_active":"2024-09-10T18:40:20.006508+0000","last_peered":"2024-09-10T18:40:20.006508+0000","last_clean":"2024-09-10T18:40:20.006508+0000","last_became_active":"2024-09-10T18:40:02.642801+0000","last_became_peered":"2024-09-10T18:40:02.642801+0000","last_unstale":"2024-09-10T18:40:20.006508+0000","last_undegraded":"2024-09-10T18:40:20.006508+0000","last_fullsized":"2024-09-10T18:40:20.006508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:53.511653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244063+0000","last_change":"2024-09-10T18:39:58.182753+0000","last_active":"2024-09-10T18:40:20.244063+0000","last_peered":"2024-09-10T18:40:20.244063+0000","last_clean":"2024-09-10T18:40:20.244063+0000","last_became_active":"2024-09-10T18:39:58.182468+0000","last_became_peered":"2024-09-10T18:39:58.182468+0000","last_unstale":"2024-09-10T18:40:20.244063+0000","last_undegraded":"2024-09-10T18:40:20.244063+0000","last_fullsized":"2024-09-10T18:40:20.244063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:42.690994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.242875+0000","last_change":"2024-09-10T18:40:01.197660+0000","last_active":"2024-09-10T18:40:20.242875+0000","last_peered":"2024-09-10T18:40:20.242875+0000","last_clean":"2024-09-10T18:40:20.242875+0000","last_became_active":"2024-09-10T18:39:56.172423+0000","last_became_peered":"2024-09-10T18:39:56.172423+0000","last_unstale":"2024-09-10T18:40:20.242875+0000","last_undegraded":"2024-09-10T18:40:20.242875+0000","last_fullsized":"2024-09-10T18:40:20.242875+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:21.606985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083450500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006304+0000","last_change":"2024-09-10T18:39:56.170689+0000","last_active":"2024-09-10T18:40:20.006304+0000","last_peered":"2024-09-10T18:40:20.006304+0000","last_clean":"2024-09-10T18:40:20.006304+0000","last_became_active":"2024-09-10T18:39:56.170428+0000","last_became_peered":"2024-09-10T18:39:56.170428+0000","last_unstale":"2024-09-10T18:40:20.006304+0000","last_undegraded":"2024-09-10T18:40:20.006304+0000","last_fullsized":"2024-09-10T18:40:20.006304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:19.461425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244486+0000","last_change":"2024-09-10T18:40:00.203178+0000","last_active":"2024-09-10T18:40:20.244486+0000","last_peered":"2024-09-10T18:40:20.244486+0000","last_clean":"2024-09-10T18:40:20.244486+0000","last_became_active":"2024-09-10T18:40:00.202894+0000","last_became_peered":"2024-09-10T18:40:00.202894+0000","last_unstale":"2024-09-10T18:40:20.244486+0000","last_undegraded":"2024-09-10T18:40:20.244486+0000","last_fullsized":"2024-09-10T18:40:20.244486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:20.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011872+0000","last_change":"2024-09-10T18:40:02.218061+0000","last_active":"2024-09-10T18:40:20.011872+0000","last_peered":"2024-09-10T18:40:20.011872+0000","last_clean":"2024-09-10T18:40:20.011872+0000","last_became_active":"2024-09-10T18:40:02.217811+0000","last_became_peered":"2024-09-10T18:40:02.217811+0000","last_unstale":"2024-09-10T18:40:20.011872+0000","last_undegraded":"2024-09-10T18:40:20.011872+0000","last_fullsized":"2024-09-10T18:40:20.011872+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:05.193655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008523+0000","last_change":"2024-09-10T18:39:58.188235+0000","last_active":"2024-09-10T18:40:20.008523+0000","last_peered":"2024-09-10T18:40:20.008523+0000","last_clean":"2024-09-10T18:40:20.008523+0000","last_became_active":"2024-09-10T18:39:58.187971+0000","last_became_peered":"2024-09-10T18:39:58.187971+0000","last_unstale":"2024-09-10T18:40:20.008523+0000","last_undegraded":"2024-09-10T18:40:20.008523+0000","last_fullsized":"2024-09-10T18:40:20.008523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:33.946776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007502+0000","last_change":"2024-09-10T18:40:01.195073+0000","last_active":"2024-09-10T18:40:20.007502+0000","last_peered":"2024-09-10T18:40:20.007502+0000","last_clean":"2024-09-10T18:40:20.007502+0000","last_became_active":"2024-09-10T18:39:56.171622+0000","last_became_peered":"2024-09-10T18:39:56.171622+0000","last_unstale":"2024-09-10T18:40:20.007502+0000","last_undegraded":"2024-09-10T18:40:20.007502+0000","last_fullsized":"2024-09-10T18:40:20.007502+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:21.137605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034161899999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243080+0000","last_change":"2024-09-10T18:39:56.173134+0000","last_active":"2024-09-10T18:40:20.243080+0000","last_peered":"2024-09-10T18:40:20.243080+0000","last_clean":"2024-09-10T18:40:20.243080+0000","last_became_active":"2024-09-10T18:39:56.172764+0000","last_became_peered":"2024-09-10T18:39:56.172764+0000","last_unstale":"2024-09-10T18:40:20.243080+0000","last_undegraded":"2024-09-10T18:40:20.243080+0000","last_fullsized":"2024-09-10T18:40:20.243080+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:46.988081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":273,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:40:38.709571+0000","last_change":"2024-09-10T18:39:47.991205+0000","last_active":"2024-09-10T18:40:38.709571+0000","last_peered":"2024-09-10T18:40:38.709571+0000","last_clean":"2024-09-10T18:40:38.709571+0000","last_became_active":"2024-09-10T18:39:47.990908+0000","last_became_peered":"2024-09-10T18:39:47.990908+0000","last_unstale":"2024-09-10T18:40:38.709571+0000","last_undegraded":"2024-09-10T18:40:38.709571+0000","last_fullsized":"2024-09-10T18:40:38.709571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:37:39.331017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:37:39.331017+0000","last_clean_scrub_stamp":"2024-09-10T18:37:39.331017+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:58.489157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006486+0000","last_change":"2024-09-10T18:40:00.201459+0000","last_active":"2024-09-10T18:40:20.006486+0000","last_peered":"2024-09-10T18:40:20.006486+0000","last_clean":"2024-09-10T18:40:20.006486+0000","last_became_active":"2024-09-10T18:40:00.201110+0000","last_became_peered":"2024-09-10T18:40:00.201110+0000","last_unstale":"2024-09-10T18:40:20.006486+0000","last_undegraded":"2024-09-10T18:40:20.006486+0000","last_fullsized":"2024-09-10T18:40:20.006486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:20.560962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244562+0000","last_change":"2024-09-10T18:40:02.218501+0000","last_active":"2024-09-10T18:40:20.244562+0000","last_peered":"2024-09-10T18:40:20.244562+0000","last_clean":"2024-09-10T18:40:20.244562+0000","last_became_active":"2024-09-10T18:40:02.218241+0000","last_became_peered":"2024-09-10T18:40:02.218241+0000","last_unstale":"2024-09-10T18:40:20.244562+0000","last_undegraded":"2024-09-10T18:40:20.244562+0000","last_fullsized":"2024-09-10T18:40:20.244562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:26.856011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":79,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.019754+0000","last_change":"2024-09-10T18:40:20.019451+0000","last_active":"2024-09-10T18:40:20.019754+0000","last_peered":"2024-09-10T18:40:20.019754+0000","last_clean":"2024-09-10T18:40:20.019754+0000","last_became_active":"2024-09-10T18:40:20.010492+0000","last_became_peered":"2024-09-10T18:40:20.010492+0000","last_unstale":"2024-09-10T18:40:20.019754+0000","last_undegraded":"2024-09-10T18:40:20.019754+0000","last_fullsized":"2024-09-10T18:40:20.019754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:29.006838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006939+0000","last_change":"2024-09-10T18:39:56.168993+0000","last_active":"2024-09-10T18:40:20.006939+0000","last_peered":"2024-09-10T18:40:20.006939+0000","last_clean":"2024-09-10T18:40:20.006939+0000","last_became_active":"2024-09-10T18:39:56.168711+0000","last_became_peered":"2024-09-10T18:39:56.168711+0000","last_unstale":"2024-09-10T18:40:20.006939+0000","last_undegraded":"2024-09-10T18:40:20.006939+0000","last_fullsized":"2024-09-10T18:40:20.006939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:14.743897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005994+0000","last_change":"2024-09-10T18:40:00.200285+0000","last_active":"2024-09-10T18:40:20.005994+0000","last_peered":"2024-09-10T18:40:20.005994+0000","last_clean":"2024-09-10T18:40:20.005994+0000","last_became_active":"2024-09-10T18:40:00.200000+0000","last_became_peered":"2024-09-10T18:40:00.200000+0000","last_unstale":"2024-09-10T18:40:20.005994+0000","last_undegraded":"2024-09-10T18:40:20.005994+0000","last_fullsized":"2024-09-10T18:40:20.005994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:08.721516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006238+0000","last_change":"2024-09-10T18:40:02.218109+0000","last_active":"2024-09-10T18:40:20.006238+0000","last_peered":"2024-09-10T18:40:20.006238+0000","last_clean":"2024-09-10T18:40:20.006238+0000","last_became_active":"2024-09-10T18:40:02.217802+0000","last_became_peered":"2024-09-10T18:40:02.217802+0000","last_unstale":"2024-09-10T18:40:20.006238+0000","last_undegraded":"2024-09-10T18:40:20.006238+0000","last_fullsized":"2024-09-10T18:40:20.006238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:35.526649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244043+0000","last_change":"2024-09-10T18:39:58.188479+0000","last_active":"2024-09-10T18:40:20.244043+0000","last_peered":"2024-09-10T18:40:20.244043+0000","last_clean":"2024-09-10T18:40:20.244043+0000","last_became_active":"2024-09-10T18:39:58.188185+0000","last_became_peered":"2024-09-10T18:39:58.188185+0000","last_unstale":"2024-09-10T18:40:20.244043+0000","last_undegraded":"2024-09-10T18:40:20.244043+0000","last_fullsized":"2024-09-10T18:40:20.244043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:27.145839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011780+0000","last_change":"2024-09-10T18:39:56.173182+0000","last_active":"2024-09-10T18:40:20.011780+0000","last_peered":"2024-09-10T18:40:20.011780+0000","last_clean":"2024-09-10T18:40:20.011780+0000","last_became_active":"2024-09-10T18:39:56.172836+0000","last_became_peered":"2024-09-10T18:39:56.172836+0000","last_unstale":"2024-09-10T18:40:20.011780+0000","last_undegraded":"2024-09-10T18:40:20.011780+0000","last_fullsized":"2024-09-10T18:40:20.011780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:35.337251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006252+0000","last_change":"2024-09-10T18:40:00.193028+0000","last_active":"2024-09-10T18:40:20.006252+0000","last_peered":"2024-09-10T18:40:20.006252+0000","last_clean":"2024-09-10T18:40:20.006252+0000","last_became_active":"2024-09-10T18:40:00.192683+0000","last_became_peered":"2024-09-10T18:40:00.192683+0000","last_unstale":"2024-09-10T18:40:20.006252+0000","last_undegraded":"2024-09-10T18:40:20.006252+0000","last_fullsized":"2024-09-10T18:40:20.006252+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:55.254130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243852+0000","last_change":"2024-09-10T18:40:02.218043+0000","last_active":"2024-09-10T18:40:20.243852+0000","last_peered":"2024-09-10T18:40:20.243852+0000","last_clean":"2024-09-10T18:40:20.243852+0000","last_became_active":"2024-09-10T18:40:02.217763+0000","last_became_peered":"2024-09-10T18:40:02.217763+0000","last_unstale":"2024-09-10T18:40:20.243852+0000","last_undegraded":"2024-09-10T18:40:20.243852+0000","last_fullsized":"2024-09-10T18:40:20.243852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:27.263161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243975+0000","last_change":"2024-09-10T18:39:58.186251+0000","last_active":"2024-09-10T18:40:20.243975+0000","last_peered":"2024-09-10T18:40:20.243975+0000","last_clean":"2024-09-10T18:40:20.243975+0000","last_became_active":"2024-09-10T18:39:58.185982+0000","last_became_peered":"2024-09-10T18:39:58.185982+0000","last_unstale":"2024-09-10T18:40:20.243975+0000","last_undegraded":"2024-09-10T18:40:20.243975+0000","last_fullsized":"2024-09-10T18:40:20.243975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:23.438037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011078+0000","last_change":"2024-09-10T18:40:20.010397+0000","last_active":"2024-09-10T18:40:20.011078+0000","last_peered":"2024-09-10T18:40:20.011078+0000","last_clean":"2024-09-10T18:40:20.011078+0000","last_became_active":"2024-09-10T18:40:20.010069+0000","last_became_peered":"2024-09-10T18:40:20.010069+0000","last_unstale":"2024-09-10T18:40:20.011078+0000","last_undegraded":"2024-09-10T18:40:20.011078+0000","last_fullsized":"2024-09-10T18:40:20.011078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:05.026738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295102+0000","last_change":"2024-09-10T18:40:00.202488+0000","last_active":"2024-09-10T18:41:19.295102+0000","last_peered":"2024-09-10T18:41:19.295102+0000","last_clean":"2024-09-10T18:41:19.295102+0000","last_became_active":"2024-09-10T18:40:00.202190+0000","last_became_peered":"2024-09-10T18:40:00.202190+0000","last_unstale":"2024-09-10T18:41:19.295102+0000","last_undegraded":"2024-09-10T18:41:19.295102+0000","last_fullsized":"2024-09-10T18:41:19.295102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:34.248722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244085+0000","last_change":"2024-09-10T18:40:02.643228+0000","last_active":"2024-09-10T18:40:20.244085+0000","last_peered":"2024-09-10T18:40:20.244085+0000","last_clean":"2024-09-10T18:40:20.244085+0000","last_became_active":"2024-09-10T18:40:02.642890+0000","last_became_peered":"2024-09-10T18:40:02.642890+0000","last_unstale":"2024-09-10T18:40:20.244085+0000","last_undegraded":"2024-09-10T18:40:20.244085+0000","last_fullsized":"2024-09-10T18:40:20.244085+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:40.705477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008575+0000","last_change":"2024-09-10T18:39:58.189049+0000","last_active":"2024-09-10T18:40:20.008575+0000","last_peered":"2024-09-10T18:40:20.008575+0000","last_clean":"2024-09-10T18:40:20.008575+0000","last_became_active":"2024-09-10T18:39:58.187544+0000","last_became_peered":"2024-09-10T18:39:58.187544+0000","last_unstale":"2024-09-10T18:40:20.008575+0000","last_undegraded":"2024-09-10T18:40:20.008575+0000","last_fullsized":"2024-09-10T18:40:20.008575+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:45.075380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009517+0000","last_change":"2024-09-10T18:39:56.169695+0000","last_active":"2024-09-10T18:40:20.009517+0000","last_peered":"2024-09-10T18:40:20.009517+0000","last_clean":"2024-09-10T18:40:20.009517+0000","last_became_active":"2024-09-10T18:39:56.169408+0000","last_became_peered":"2024-09-10T18:39:56.169408+0000","last_unstale":"2024-09-10T18:40:20.009517+0000","last_undegraded":"2024-09-10T18:40:20.009517+0000","last_fullsized":"2024-09-10T18:40:20.009517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:52.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005911+0000","last_change":"2024-09-10T18:40:00.197321+0000","last_active":"2024-09-10T18:40:20.005911+0000","last_peered":"2024-09-10T18:40:20.005911+0000","last_clean":"2024-09-10T18:40:20.005911+0000","last_became_active":"2024-09-10T18:40:00.197025+0000","last_became_peered":"2024-09-10T18:40:00.197025+0000","last_unstale":"2024-09-10T18:40:20.005911+0000","last_undegraded":"2024-09-10T18:40:20.005911+0000","last_fullsized":"2024-09-10T18:40:20.005911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:46.190746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007524+0000","last_change":"2024-09-10T18:40:02.215551+0000","last_active":"2024-09-10T18:40:20.007524+0000","last_peered":"2024-09-10T18:40:20.007524+0000","last_clean":"2024-09-10T18:40:20.007524+0000","last_became_active":"2024-09-10T18:40:02.215223+0000","last_became_peered":"2024-09-10T18:40:02.215223+0000","last_unstale":"2024-09-10T18:40:20.007524+0000","last_undegraded":"2024-09-10T18:40:20.007524+0000","last_fullsized":"2024-09-10T18:40:20.007524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:18.424945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244029+0000","last_change":"2024-09-10T18:39:58.184914+0000","last_active":"2024-09-10T18:40:20.244029+0000","last_peered":"2024-09-10T18:40:20.244029+0000","last_clean":"2024-09-10T18:40:20.244029+0000","last_became_active":"2024-09-10T18:39:58.184624+0000","last_became_peered":"2024-09-10T18:39:58.184624+0000","last_unstale":"2024-09-10T18:40:20.244029+0000","last_undegraded":"2024-09-10T18:40:20.244029+0000","last_fullsized":"2024-09-10T18:40:20.244029+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:42.937378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005970+0000","last_change":"2024-09-10T18:39:56.175019+0000","last_active":"2024-09-10T18:40:20.005970+0000","last_peered":"2024-09-10T18:40:20.005970+0000","last_clean":"2024-09-10T18:40:20.005970+0000","last_became_active":"2024-09-10T18:39:56.174685+0000","last_became_peered":"2024-09-10T18:39:56.174685+0000","last_unstale":"2024-09-10T18:40:20.005970+0000","last_undegraded":"2024-09-10T18:40:20.005970+0000","last_fullsized":"2024-09-10T18:40:20.005970+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:07.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008092+0000","last_change":"2024-09-10T18:40:00.198765+0000","last_active":"2024-09-10T18:40:20.008092+0000","last_peered":"2024-09-10T18:40:20.008092+0000","last_clean":"2024-09-10T18:40:20.008092+0000","last_became_active":"2024-09-10T18:40:00.198498+0000","last_became_peered":"2024-09-10T18:40:00.198498+0000","last_unstale":"2024-09-10T18:40:20.008092+0000","last_undegraded":"2024-09-10T18:40:20.008092+0000","last_fullsized":"2024-09-10T18:40:20.008092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:57.276713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006374+0000","last_change":"2024-09-10T18:40:02.221139+0000","last_active":"2024-09-10T18:40:20.006374+0000","last_peered":"2024-09-10T18:40:20.006374+0000","last_clean":"2024-09-10T18:40:20.006374+0000","last_became_active":"2024-09-10T18:40:02.220858+0000","last_became_peered":"2024-09-10T18:40:02.220858+0000","last_unstale":"2024-09-10T18:40:20.006374+0000","last_undegraded":"2024-09-10T18:40:20.006374+0000","last_fullsized":"2024-09-10T18:40:20.006374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:27.534137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008228+0000","last_change":"2024-09-10T18:39:56.167572+0000","last_active":"2024-09-10T18:40:20.008228+0000","last_peered":"2024-09-10T18:40:20.008228+0000","last_clean":"2024-09-10T18:40:20.008228+0000","last_became_active":"2024-09-10T18:39:56.167283+0000","last_became_peered":"2024-09-10T18:39:56.167283+0000","last_unstale":"2024-09-10T18:40:20.008228+0000","last_undegraded":"2024-09-10T18:40:20.008228+0000","last_fullsized":"2024-09-10T18:40:20.008228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:04.731029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008178+0000","last_change":"2024-09-10T18:39:58.182072+0000","last_active":"2024-09-10T18:40:20.008178+0000","last_peered":"2024-09-10T18:40:20.008178+0000","last_clean":"2024-09-10T18:40:20.008178+0000","last_became_active":"2024-09-10T18:39:58.181824+0000","last_became_peered":"2024-09-10T18:39:58.181824+0000","last_unstale":"2024-09-10T18:40:20.008178+0000","last_undegraded":"2024-09-10T18:40:20.008178+0000","last_fullsized":"2024-09-10T18:40:20.008178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:30.966225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244472+0000","last_change":"2024-09-10T18:40:00.203124+0000","last_active":"2024-09-10T18:40:20.244472+0000","last_peered":"2024-09-10T18:40:20.244472+0000","last_clean":"2024-09-10T18:40:20.244472+0000","last_became_active":"2024-09-10T18:40:00.202829+0000","last_became_peered":"2024-09-10T18:40:00.202829+0000","last_unstale":"2024-09-10T18:40:20.244472+0000","last_undegraded":"2024-09-10T18:40:20.244472+0000","last_fullsized":"2024-09-10T18:40:20.244472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:12.371398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006455+0000","last_change":"2024-09-10T18:40:02.645417+0000","last_active":"2024-09-10T18:40:20.006455+0000","last_peered":"2024-09-10T18:40:20.006455+0000","last_clean":"2024-09-10T18:40:20.006455+0000","last_became_active":"2024-09-10T18:40:02.645136+0000","last_became_peered":"2024-09-10T18:40:02.645136+0000","last_unstale":"2024-09-10T18:40:20.006455+0000","last_undegraded":"2024-09-10T18:40:20.006455+0000","last_fullsized":"2024-09-10T18:40:20.006455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:30.247254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007666+0000","last_change":"2024-09-10T18:39:58.182046+0000","last_active":"2024-09-10T18:40:20.007666+0000","last_peered":"2024-09-10T18:40:20.007666+0000","last_clean":"2024-09-10T18:40:20.007666+0000","last_became_active":"2024-09-10T18:39:58.181566+0000","last_became_peered":"2024-09-10T18:39:58.181566+0000","last_unstale":"2024-09-10T18:40:20.007666+0000","last_undegraded":"2024-09-10T18:40:20.007666+0000","last_fullsized":"2024-09-10T18:40:20.007666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:16.291634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243397+0000","last_change":"2024-09-10T18:39:56.173580+0000","last_active":"2024-09-10T18:40:20.243397+0000","last_peered":"2024-09-10T18:40:20.243397+0000","last_clean":"2024-09-10T18:40:20.243397+0000","last_became_active":"2024-09-10T18:39:56.173314+0000","last_became_peered":"2024-09-10T18:39:56.173314+0000","last_unstale":"2024-09-10T18:40:20.243397+0000","last_undegraded":"2024-09-10T18:40:20.243397+0000","last_fullsized":"2024-09-10T18:40:20.243397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:14.122137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295121+0000","last_change":"2024-09-10T18:40:00.203886+0000","last_active":"2024-09-10T18:41:19.295121+0000","last_peered":"2024-09-10T18:41:19.295121+0000","last_clean":"2024-09-10T18:41:19.295121+0000","last_became_active":"2024-09-10T18:40:00.203601+0000","last_became_peered":"2024-09-10T18:40:00.203601+0000","last_unstale":"2024-09-10T18:41:19.295121+0000","last_undegraded":"2024-09-10T18:41:19.295121+0000","last_fullsized":"2024-09-10T18:41:19.295121+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:03.938604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243877+0000","last_change":"2024-09-10T18:40:02.221141+0000","last_active":"2024-09-10T18:40:20.243877+0000","last_peered":"2024-09-10T18:40:20.243877+0000","last_clean":"2024-09-10T18:40:20.243877+0000","last_became_active":"2024-09-10T18:40:02.220850+0000","last_became_peered":"2024-09-10T18:40:02.220850+0000","last_unstale":"2024-09-10T18:40:20.243877+0000","last_undegraded":"2024-09-10T18:40:20.243877+0000","last_fullsized":"2024-09-10T18:40:20.243877+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:43.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244873+0000","last_change":"2024-09-10T18:39:58.184494+0000","last_active":"2024-09-10T18:40:20.244873+0000","last_peered":"2024-09-10T18:40:20.244873+0000","last_clean":"2024-09-10T18:40:20.244873+0000","last_became_active":"2024-09-10T18:39:58.184244+0000","last_became_peered":"2024-09-10T18:39:58.184244+0000","last_unstale":"2024-09-10T18:40:20.244873+0000","last_undegraded":"2024-09-10T18:40:20.244873+0000","last_fullsized":"2024-09-10T18:40:20.244873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:40.137215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243202+0000","last_change":"2024-09-10T18:39:56.172917+0000","last_active":"2024-09-10T18:40:20.243202+0000","last_peered":"2024-09-10T18:40:20.243202+0000","last_clean":"2024-09-10T18:40:20.243202+0000","last_became_active":"2024-09-10T18:39:56.172634+0000","last_became_peered":"2024-09-10T18:39:56.172634+0000","last_unstale":"2024-09-10T18:40:20.243202+0000","last_undegraded":"2024-09-10T18:40:20.243202+0000","last_fullsized":"2024-09-10T18:40:20.243202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:01.452156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294929+0000","last_change":"2024-09-10T18:40:00.199630+0000","last_active":"2024-09-10T18:41:19.294929+0000","last_peered":"2024-09-10T18:41:19.294929+0000","last_clean":"2024-09-10T18:41:19.294929+0000","last_became_active":"2024-09-10T18:40:00.199354+0000","last_became_peered":"2024-09-10T18:40:00.199354+0000","last_unstale":"2024-09-10T18:41:19.294929+0000","last_undegraded":"2024-09-10T18:41:19.294929+0000","last_fullsized":"2024-09-10T18:41:19.294929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:37.611666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009405+0000","last_change":"2024-09-10T18:40:02.217502+0000","last_active":"2024-09-10T18:40:20.009405+0000","last_peered":"2024-09-10T18:40:20.009405+0000","last_clean":"2024-09-10T18:40:20.009405+0000","last_became_active":"2024-09-10T18:40:02.217242+0000","last_became_peered":"2024-09-10T18:40:02.217242+0000","last_unstale":"2024-09-10T18:40:20.009405+0000","last_undegraded":"2024-09-10T18:40:20.009405+0000","last_fullsized":"2024-09-10T18:40:20.009405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:13.097081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010980+0000","last_change":"2024-09-10T18:39:58.189204+0000","last_active":"2024-09-10T18:40:20.010980+0000","last_peered":"2024-09-10T18:40:20.010980+0000","last_clean":"2024-09-10T18:40:20.010980+0000","last_became_active":"2024-09-10T18:39:58.188532+0000","last_became_peered":"2024-09-10T18:39:58.188532+0000","last_unstale":"2024-09-10T18:40:20.010980+0000","last_undegraded":"2024-09-10T18:40:20.010980+0000","last_fullsized":"2024-09-10T18:40:20.010980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:31.999746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008327+0000","last_change":"2024-09-10T18:39:56.174627+0000","last_active":"2024-09-10T18:40:20.008327+0000","last_peered":"2024-09-10T18:40:20.008327+0000","last_clean":"2024-09-10T18:40:20.008327+0000","last_became_active":"2024-09-10T18:39:56.174369+0000","last_became_peered":"2024-09-10T18:39:56.174369+0000","last_unstale":"2024-09-10T18:40:20.008327+0000","last_undegraded":"2024-09-10T18:40:20.008327+0000","last_fullsized":"2024-09-10T18:40:20.008327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:19.919451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007595+0000","last_change":"2024-09-10T18:40:00.203117+0000","last_active":"2024-09-10T18:40:20.007595+0000","last_peered":"2024-09-10T18:40:20.007595+0000","last_clean":"2024-09-10T18:40:20.007595+0000","last_became_active":"2024-09-10T18:40:00.202662+0000","last_became_peered":"2024-09-10T18:40:00.202662+0000","last_unstale":"2024-09-10T18:40:20.007595+0000","last_undegraded":"2024-09-10T18:40:20.007595+0000","last_fullsized":"2024-09-10T18:40:20.007595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:08.271516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243290+0000","last_change":"2024-09-10T18:40:02.221280+0000","last_active":"2024-09-10T18:40:20.243290+0000","last_peered":"2024-09-10T18:40:20.243290+0000","last_clean":"2024-09-10T18:40:20.243290+0000","last_became_active":"2024-09-10T18:40:02.221037+0000","last_became_peered":"2024-09-10T18:40:02.221037+0000","last_unstale":"2024-09-10T18:40:20.243290+0000","last_undegraded":"2024-09-10T18:40:20.243290+0000","last_fullsized":"2024-09-10T18:40:20.243290+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.997975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009182+0000","last_change":"2024-09-10T18:39:58.189088+0000","last_active":"2024-09-10T18:40:20.009182+0000","last_peered":"2024-09-10T18:40:20.009182+0000","last_clean":"2024-09-10T18:40:20.009182+0000","last_became_active":"2024-09-10T18:39:58.188846+0000","last_became_peered":"2024-09-10T18:39:58.188846+0000","last_unstale":"2024-09-10T18:40:20.009182+0000","last_undegraded":"2024-09-10T18:40:20.009182+0000","last_fullsized":"2024-09-10T18:40:20.009182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:19.568943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011709+0000","last_change":"2024-09-10T18:39:56.168804+0000","last_active":"2024-09-10T18:40:20.011709+0000","last_peered":"2024-09-10T18:40:20.011709+0000","last_clean":"2024-09-10T18:40:20.011709+0000","last_became_active":"2024-09-10T18:39:56.168528+0000","last_became_peered":"2024-09-10T18:39:56.168528+0000","last_unstale":"2024-09-10T18:40:20.011709+0000","last_undegraded":"2024-09-10T18:40:20.011709+0000","last_fullsized":"2024-09-10T18:40:20.011709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:45.447696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010878+0000","last_change":"2024-09-10T18:40:00.199892+0000","last_active":"2024-09-10T18:40:20.010878+0000","last_peered":"2024-09-10T18:40:20.010878+0000","last_clean":"2024-09-10T18:40:20.010878+0000","last_became_active":"2024-09-10T18:40:00.199568+0000","last_became_peered":"2024-09-10T18:40:00.199568+0000","last_unstale":"2024-09-10T18:40:20.010878+0000","last_undegraded":"2024-09-10T18:40:20.010878+0000","last_fullsized":"2024-09-10T18:40:20.010878+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:07.598997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006570+0000","last_change":"2024-09-10T18:40:02.218612+0000","last_active":"2024-09-10T18:40:20.006570+0000","last_peered":"2024-09-10T18:40:20.006570+0000","last_clean":"2024-09-10T18:40:20.006570+0000","last_became_active":"2024-09-10T18:40:02.218310+0000","last_became_peered":"2024-09-10T18:40:02.218310+0000","last_unstale":"2024-09-10T18:40:20.006570+0000","last_undegraded":"2024-09-10T18:40:20.006570+0000","last_fullsized":"2024-09-10T18:40:20.006570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:33.599593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.022089+0000","last_change":"2024-09-10T18:40:20.021794+0000","last_active":"2024-09-10T18:40:20.022089+0000","last_peered":"2024-09-10T18:40:20.022089+0000","last_clean":"2024-09-10T18:40:20.022089+0000","last_became_active":"2024-09-10T18:40:20.010027+0000","last_became_peered":"2024-09-10T18:40:20.010027+0000","last_unstale":"2024-09-10T18:40:20.022089+0000","last_undegraded":"2024-09-10T18:40:20.022089+0000","last_fullsized":"2024-09-10T18:40:20.022089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:08.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243127+0000","last_change":"2024-09-10T18:39:56.168677+0000","last_active":"2024-09-10T18:40:20.243127+0000","last_peered":"2024-09-10T18:40:20.243127+0000","last_clean":"2024-09-10T18:40:20.243127+0000","last_became_active":"2024-09-10T18:39:56.168376+0000","last_became_peered":"2024-09-10T18:39:56.168376+0000","last_unstale":"2024-09-10T18:40:20.243127+0000","last_undegraded":"2024-09-10T18:40:20.243127+0000","last_fullsized":"2024-09-10T18:40:20.243127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:46.683638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008468+0000","last_change":"2024-09-10T18:40:00.205229+0000","last_active":"2024-09-10T18:40:20.008468+0000","last_peered":"2024-09-10T18:40:20.008468+0000","last_clean":"2024-09-10T18:40:20.008468+0000","last_became_active":"2024-09-10T18:40:00.204967+0000","last_became_peered":"2024-09-10T18:40:00.204967+0000","last_unstale":"2024-09-10T18:40:20.008468+0000","last_undegraded":"2024-09-10T18:40:20.008468+0000","last_fullsized":"2024-09-10T18:40:20.008468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:11.921294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006154+0000","last_change":"2024-09-10T18:40:02.216361+0000","last_active":"2024-09-10T18:40:20.006154+0000","last_peered":"2024-09-10T18:40:20.006154+0000","last_clean":"2024-09-10T18:40:20.006154+0000","last_became_active":"2024-09-10T18:40:02.216055+0000","last_became_peered":"2024-09-10T18:40:02.216055+0000","last_unstale":"2024-09-10T18:40:20.006154+0000","last_undegraded":"2024-09-10T18:40:20.006154+0000","last_fullsized":"2024-09-10T18:40:20.006154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:26.629430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007152+0000","last_change":"2024-09-10T18:39:58.187272+0000","last_active":"2024-09-10T18:40:20.007152+0000","last_peered":"2024-09-10T18:40:20.007152+0000","last_clean":"2024-09-10T18:40:20.007152+0000","last_became_active":"2024-09-10T18:39:58.186202+0000","last_became_peered":"2024-09-10T18:39:58.186202+0000","last_unstale":"2024-09-10T18:40:20.007152+0000","last_undegraded":"2024-09-10T18:40:20.007152+0000","last_fullsized":"2024-09-10T18:40:20.007152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:19.241984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008635+0000","last_change":"2024-09-10T18:39:56.173502+0000","last_active":"2024-09-10T18:40:20.008635+0000","last_peered":"2024-09-10T18:40:20.008635+0000","last_clean":"2024-09-10T18:40:20.008635+0000","last_became_active":"2024-09-10T18:39:56.173248+0000","last_became_peered":"2024-09-10T18:39:56.173248+0000","last_unstale":"2024-09-10T18:40:20.008635+0000","last_undegraded":"2024-09-10T18:40:20.008635+0000","last_fullsized":"2024-09-10T18:40:20.008635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:28.812714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005992+0000","last_change":"2024-09-10T18:40:00.202354+0000","last_active":"2024-09-10T18:40:20.005992+0000","last_peered":"2024-09-10T18:40:20.005992+0000","last_clean":"2024-09-10T18:40:20.005992+0000","last_became_active":"2024-09-10T18:40:00.202030+0000","last_became_peered":"2024-09-10T18:40:00.202030+0000","last_unstale":"2024-09-10T18:40:20.005992+0000","last_undegraded":"2024-09-10T18:40:20.005992+0000","last_fullsized":"2024-09-10T18:40:20.005992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:22.805489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007445+0000","last_change":"2024-09-10T18:40:02.643876+0000","last_active":"2024-09-10T18:40:20.007445+0000","last_peered":"2024-09-10T18:40:20.007445+0000","last_clean":"2024-09-10T18:40:20.007445+0000","last_became_active":"2024-09-10T18:40:02.643249+0000","last_became_peered":"2024-09-10T18:40:02.643249+0000","last_unstale":"2024-09-10T18:40:20.007445+0000","last_undegraded":"2024-09-10T18:40:20.007445+0000","last_fullsized":"2024-09-10T18:40:20.007445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:22.913203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006417+0000","last_change":"2024-09-10T18:40:02.222541+0000","last_active":"2024-09-10T18:40:20.006417+0000","last_peered":"2024-09-10T18:40:20.006417+0000","last_clean":"2024-09-10T18:40:20.006417+0000","last_became_active":"2024-09-10T18:40:02.222280+0000","last_became_peered":"2024-09-10T18:40:02.222280+0000","last_unstale":"2024-09-10T18:40:20.006417+0000","last_undegraded":"2024-09-10T18:40:20.006417+0000","last_fullsized":"2024-09-10T18:40:20.006417+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:40.368982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243061+0000","last_change":"2024-09-10T18:39:56.172788+0000","last_active":"2024-09-10T18:40:20.243061+0000","last_peered":"2024-09-10T18:40:20.243061+0000","last_clean":"2024-09-10T18:40:20.243061+0000","last_became_active":"2024-09-10T18:39:56.172543+0000","last_became_peered":"2024-09-10T18:39:56.172543+0000","last_unstale":"2024-09-10T18:40:20.243061+0000","last_undegraded":"2024-09-10T18:40:20.243061+0000","last_fullsized":"2024-09-10T18:40:20.243061+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.211986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243977+0000","last_change":"2024-09-10T18:39:58.185397+0000","last_active":"2024-09-10T18:40:20.243977+0000","last_peered":"2024-09-10T18:40:20.243977+0000","last_clean":"2024-09-10T18:40:20.243977+0000","last_became_active":"2024-09-10T18:39:58.185088+0000","last_became_peered":"2024-09-10T18:39:58.185088+0000","last_unstale":"2024-09-10T18:40:20.243977+0000","last_undegraded":"2024-09-10T18:40:20.243977+0000","last_fullsized":"2024-09-10T18:40:20.243977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:21.988849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295073+0000","last_change":"2024-09-10T18:40:00.208003+0000","last_active":"2024-09-10T18:41:19.295073+0000","last_peered":"2024-09-10T18:41:19.295073+0000","last_clean":"2024-09-10T18:41:19.295073+0000","last_became_active":"2024-09-10T18:40:00.206620+0000","last_became_peered":"2024-09-10T18:40:00.206620+0000","last_unstale":"2024-09-10T18:41:19.295073+0000","last_undegraded":"2024-09-10T18:41:19.295073+0000","last_fullsized":"2024-09-10T18:41:19.295073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:48.044558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011332+0000","last_change":"2024-09-10T18:40:02.218949+0000","last_active":"2024-09-10T18:40:20.011332+0000","last_peered":"2024-09-10T18:40:20.011332+0000","last_clean":"2024-09-10T18:40:20.011332+0000","last_became_active":"2024-09-10T18:40:02.218557+0000","last_became_peered":"2024-09-10T18:40:02.218557+0000","last_unstale":"2024-09-10T18:40:20.011332+0000","last_undegraded":"2024-09-10T18:40:20.011332+0000","last_fullsized":"2024-09-10T18:40:20.011332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:14.161856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"60'1","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007315+0000","last_change":"2024-09-10T18:39:56.172799+0000","last_active":"2024-09-10T18:40:20.007315+0000","last_peered":"2024-09-10T18:40:20.007315+0000","last_clean":"2024-09-10T18:40:20.007315+0000","last_became_active":"2024-09-10T18:39:56.172533+0000","last_became_peered":"2024-09-10T18:39:56.172533+0000","last_unstale":"2024-09-10T18:40:20.007315+0000","last_undegraded":"2024-09-10T18:40:20.007315+0000","last_fullsized":"2024-09-10T18:40:20.007315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:14.896618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008615+0000","last_change":"2024-09-10T18:39:58.193514+0000","last_active":"2024-09-10T18:40:20.008615+0000","last_peered":"2024-09-10T18:40:20.008615+0000","last_clean":"2024-09-10T18:40:20.008615+0000","last_became_active":"2024-09-10T18:39:58.193136+0000","last_became_peered":"2024-09-10T18:39:58.193136+0000","last_unstale":"2024-09-10T18:40:20.008615+0000","last_undegraded":"2024-09-10T18:40:20.008615+0000","last_fullsized":"2024-09-10T18:40:20.008615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:26.321586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243873+0000","last_change":"2024-09-10T18:40:00.200693+0000","last_active":"2024-09-10T18:40:20.243873+0000","last_peered":"2024-09-10T18:40:20.243873+0000","last_clean":"2024-09-10T18:40:20.243873+0000","last_became_active":"2024-09-10T18:40:00.200405+0000","last_became_peered":"2024-09-10T18:40:00.200405+0000","last_unstale":"2024-09-10T18:40:20.243873+0000","last_undegraded":"2024-09-10T18:40:20.243873+0000","last_fullsized":"2024-09-10T18:40:20.243873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:39.635593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008831+0000","last_change":"2024-09-10T18:40:02.643206+0000","last_active":"2024-09-10T18:40:20.008831+0000","last_peered":"2024-09-10T18:40:20.008831+0000","last_clean":"2024-09-10T18:40:20.008831+0000","last_became_active":"2024-09-10T18:40:02.642935+0000","last_became_peered":"2024-09-10T18:40:02.642935+0000","last_unstale":"2024-09-10T18:40:20.008831+0000","last_undegraded":"2024-09-10T18:40:20.008831+0000","last_fullsized":"2024-09-10T18:40:20.008831+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:25.174406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008811+0000","last_change":"2024-09-10T18:39:56.170131+0000","last_active":"2024-09-10T18:40:20.008811+0000","last_peered":"2024-09-10T18:40:20.008811+0000","last_clean":"2024-09-10T18:40:20.008811+0000","last_became_active":"2024-09-10T18:39:56.169869+0000","last_became_peered":"2024-09-10T18:39:56.169869+0000","last_unstale":"2024-09-10T18:40:20.008811+0000","last_undegraded":"2024-09-10T18:40:20.008811+0000","last_fullsized":"2024-09-10T18:40:20.008811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:28.042823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006687+0000","last_change":"2024-09-10T18:39:58.179822+0000","last_active":"2024-09-10T18:40:20.006687+0000","last_peered":"2024-09-10T18:40:20.006687+0000","last_clean":"2024-09-10T18:40:20.006687+0000","last_became_active":"2024-09-10T18:39:58.179530+0000","last_became_peered":"2024-09-10T18:39:58.179530+0000","last_unstale":"2024-09-10T18:40:20.006687+0000","last_undegraded":"2024-09-10T18:40:20.006687+0000","last_fullsized":"2024-09-10T18:40:20.006687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:39.742732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011571+0000","last_change":"2024-09-10T18:40:00.201621+0000","last_active":"2024-09-10T18:40:20.011571+0000","last_peered":"2024-09-10T18:40:20.011571+0000","last_clean":"2024-09-10T18:40:20.011571+0000","last_became_active":"2024-09-10T18:40:00.201264+0000","last_became_peered":"2024-09-10T18:40:00.201264+0000","last_unstale":"2024-09-10T18:40:20.011571+0000","last_undegraded":"2024-09-10T18:40:20.011571+0000","last_fullsized":"2024-09-10T18:40:20.011571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:55.861590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008088+0000","last_change":"2024-09-10T18:40:02.643833+0000","last_active":"2024-09-10T18:40:20.008088+0000","last_peered":"2024-09-10T18:40:20.008088+0000","last_clean":"2024-09-10T18:40:20.008088+0000","last_became_active":"2024-09-10T18:40:02.643245+0000","last_became_peered":"2024-09-10T18:40:02.643245+0000","last_unstale":"2024-09-10T18:40:20.008088+0000","last_undegraded":"2024-09-10T18:40:20.008088+0000","last_fullsized":"2024-09-10T18:40:20.008088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:53.773097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007711+0000","last_change":"2024-09-10T18:39:56.178535+0000","last_active":"2024-09-10T18:40:20.007711+0000","last_peered":"2024-09-10T18:40:20.007711+0000","last_clean":"2024-09-10T18:40:20.007711+0000","last_became_active":"2024-09-10T18:39:56.178240+0000","last_became_peered":"2024-09-10T18:39:56.178240+0000","last_unstale":"2024-09-10T18:40:20.007711+0000","last_undegraded":"2024-09-10T18:40:20.007711+0000","last_fullsized":"2024-09-10T18:40:20.007711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:01.394833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007824+0000","last_change":"2024-09-10T18:39:58.187539+0000","last_active":"2024-09-10T18:40:20.007824+0000","last_peered":"2024-09-10T18:40:20.007824+0000","last_clean":"2024-09-10T18:40:20.007824+0000","last_became_active":"2024-09-10T18:39:58.187231+0000","last_became_peered":"2024-09-10T18:39:58.187231+0000","last_unstale":"2024-09-10T18:40:20.007824+0000","last_undegraded":"2024-09-10T18:40:20.007824+0000","last_fullsized":"2024-09-10T18:40:20.007824+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:43.864399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009194+0000","last_change":"2024-09-10T18:40:00.199135+0000","last_active":"2024-09-10T18:40:20.009194+0000","last_peered":"2024-09-10T18:40:20.009194+0000","last_clean":"2024-09-10T18:40:20.009194+0000","last_became_active":"2024-09-10T18:40:00.198871+0000","last_became_peered":"2024-09-10T18:40:00.198871+0000","last_unstale":"2024-09-10T18:40:20.009194+0000","last_undegraded":"2024-09-10T18:40:20.009194+0000","last_fullsized":"2024-09-10T18:40:20.009194+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:19.050238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234008,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298696,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442360,"statfs":{"total":95990841344,"available":95684976640,"internally_reserved":0,"allocated":1376256,"data_stored":921457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.63700000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.624},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36399999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495643,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298696,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442360,"statfs":{"total":95990841344,"available":95684976640,"internally_reserved":0,"allocated":1372160,"data_stored":922420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.628,"5min":0.753,"15min":0.753},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.443}]},{"osd":3,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48499999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73099999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.498}]},{"osd":5,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.436,"5min":0.433,"15min":0.433},"min":{"1min":0.32400000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65500000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757281,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298112,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442944,"statfs":{"total":95990841344,"available":95685574656,"internally_reserved":0,"allocated":774144,"data_stored":330862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59799999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64200000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64500000000000002,"5min":0.749,"15min":0.749},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.497,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.627,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65400000000000003,"5min":0.754,"15min":0.754},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.41499999999999998,"15min":0.40100000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68400000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46300000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018918,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":332137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.59599999999999997,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":9.4949999999999992,"5min":9.4949999999999992,"15min":9.4949999999999992},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.57299999999999995,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.435}]},{"osd":3,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.60899999999999999,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:40:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50800000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.41899999999999998,"15min":0.40799999999999997},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59399999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442904,"statfs":{"total":95990841344,"available":95685533696,"internally_reserved":0,"allocated":815104,"data_stored":331989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.317}]},{"osd":1,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.27300000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40500000000000003,"15min":0.40100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.622,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:40:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.441},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.374}]},{"osd":5,"last update":"Tue Sep 10 18:40:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442948,"statfs":{"total":95990841344,"available":95685578752,"internally_reserved":0,"allocated":770048,"data_stored":330450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.372,"15min":0.35999999999999999},"min":{"1min":0.28000000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53200000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.371,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.254,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59299999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.69699999999999995,"5min":0.751,"15min":0.751},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63100000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.434},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.436,"15min":0.432},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54300000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.503}]},{"osd":4,"last update":"Tue Sep 10 18:40:34 2024","interfaces":[{"interface":"back","average":{"1min":1.7,"5min":0.753,"15min":0.59499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":18.253,"5min":18.253,"15min":18.253},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.48699999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76200000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442904,"statfs":{"total":95990841344,"available":95685533696,"internally_reserved":0,"allocated":815104,"data_stored":332813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35299999999999998,"15min":0.33300000000000002},"min":{"1min":0.224,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.49299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.371,"15min":0.33900000000000002},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.58399999999999996,"5min":0.75,"15min":0.75},"last":0.27200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495}]},{"osd":3,"last update":"Tue Sep 10 18:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61399999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.69399999999999995,"15min":0.628},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.497999999999999,"5min":10.497999999999999,"15min":10.497999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.874,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74099999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82399999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298708,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442348,"statfs":{"total":95990841344,"available":95684964352,"internally_reserved":0,"allocated":1388544,"data_stored":921852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55300000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41299999999999998},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.626,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.378}]},{"osd":4,"last update":"Tue Sep 10 18:40:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73599999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71499999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.753,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70499999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:41:29.934 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:41:30.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:30 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:30.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:30 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:30.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:30 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:30.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:30 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:31.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:30 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:31.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:30 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:31.363 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:31.675 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:31 smithi066 bash[58568]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:41:31.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:31 smithi117 ceph-mon[27457]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:31.953 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:31 smithi066 ceph-mon[31695]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:31.954 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:31 smithi066 ceph-mon[25398]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:31.954 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:31 smithi066 bash[58568]: Writing manifest to image destination 2024-09-10T18:41:32.665 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 podman[58568]: 2024-09-10 18:41:32.286523618 +0000 UTC m=+3.723839304 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:41:32.665 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 podman[58568]: 2024-09-10 18:41:32.557535087 +0000 UTC m=+3.994850758 container create d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:33.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.229 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 podman[58568]: 2024-09-10 18:41:32.785634587 +0000 UTC m=+4.222950267 container init d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:33.229 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 podman[58568]: 2024-09-10 18:41:32.788581673 +0000 UTC m=+4.225897354 container start d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:33.229 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.792Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.792Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.792Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.792Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:41:33.230 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:41:33.231 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:41:33.232 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.793Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:41:33.233 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a[58979]: ts=2024-09-10T18:41:32.794Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 bash[58568]: d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 2024-09-10T18:41:33.234 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:41:32 smithi066 systemd[1]: Started Ceph node-exporter.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:33.249 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:32 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.249 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:32 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.249 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:32 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:33.816 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:33 smithi117 systemd[1]: Starting Ceph node-exporter.b for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:34.074 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:41:34.075 INFO:teuthology.orchestra.run.smithi066.stderr:dumped all 2024-09-10T18:41:34.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:33 smithi117 ceph-mon[27457]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:34.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:33 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.217 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:33 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.218 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:33 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.218 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:33 smithi117 ceph-mon[27457]: Deploying daemon node-exporter.b on smithi117 2024-09-10T18:41:34.218 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:33 smithi117 bash[52193]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:41:34.218 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:34 smithi117 bash[52193]: Getting image source signatures 2024-09-10T18:41:34.218 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:34 smithi117 bash[52193]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:41:34.218 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:34 smithi117 bash[52193]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[31695]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[31695]: Deploying daemon node-exporter.b on smithi117 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[25398]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:34.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:34.229 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:33 smithi066 ceph-mon[25398]: Deploying daemon node-exporter.b on smithi117 2024-09-10T18:41:34.565 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:34 smithi117 bash[52193]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:41:34.726 INFO:teuthology.orchestra.run.smithi066.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-10T18:41:32.658152+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":986,"num_read_kb":839,"num_write":609,"num_write_kb":3237,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1771324,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":634,"ondisk_log_size":634,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749928448,"kb_used":2386756,"kb_used_data":7916,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747541692,"statfs":{"total":767926730752,"available":765482692608,"internally_reserved":0,"allocated":8105984,"data_stored":4423980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002856"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007348+0000","last_change":"2024-09-10T18:40:02.643950+0000","last_active":"2024-09-10T18:40:20.007348+0000","last_peered":"2024-09-10T18:40:20.007348+0000","last_clean":"2024-09-10T18:40:20.007348+0000","last_became_active":"2024-09-10T18:40:02.643334+0000","last_became_peered":"2024-09-10T18:40:02.643334+0000","last_unstale":"2024-09-10T18:40:20.007348+0000","last_undegraded":"2024-09-10T18:40:20.007348+0000","last_fullsized":"2024-09-10T18:40:20.007348+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:20.198514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243117+0000","last_change":"2024-09-10T18:39:56.176131+0000","last_active":"2024-09-10T18:40:20.243117+0000","last_peered":"2024-09-10T18:40:20.243117+0000","last_clean":"2024-09-10T18:40:20.243117+0000","last_became_active":"2024-09-10T18:39:56.175827+0000","last_became_peered":"2024-09-10T18:39:56.175827+0000","last_unstale":"2024-09-10T18:40:20.243117+0000","last_undegraded":"2024-09-10T18:40:20.243117+0000","last_fullsized":"2024-09-10T18:40:20.243117+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:40.111091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007241+0000","last_change":"2024-09-10T18:39:58.184433+0000","last_active":"2024-09-10T18:40:20.007241+0000","last_peered":"2024-09-10T18:40:20.007241+0000","last_clean":"2024-09-10T18:40:20.007241+0000","last_became_active":"2024-09-10T18:39:58.184161+0000","last_became_peered":"2024-09-10T18:39:58.184161+0000","last_unstale":"2024-09-10T18:40:20.007241+0000","last_undegraded":"2024-09-10T18:40:20.007241+0000","last_fullsized":"2024-09-10T18:40:20.007241+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:41.953694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008153+0000","last_change":"2024-09-10T18:40:00.204158+0000","last_active":"2024-09-10T18:40:20.008153+0000","last_peered":"2024-09-10T18:40:20.008153+0000","last_clean":"2024-09-10T18:40:20.008153+0000","last_became_active":"2024-09-10T18:40:00.203865+0000","last_became_peered":"2024-09-10T18:40:00.203865+0000","last_unstale":"2024-09-10T18:40:20.008153+0000","last_undegraded":"2024-09-10T18:40:20.008153+0000","last_fullsized":"2024-09-10T18:40:20.008153+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:46.667637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008378+0000","last_change":"2024-09-10T18:39:56.178829+0000","last_active":"2024-09-10T18:40:20.008378+0000","last_peered":"2024-09-10T18:40:20.008378+0000","last_clean":"2024-09-10T18:40:20.008378+0000","last_became_active":"2024-09-10T18:39:56.178254+0000","last_became_peered":"2024-09-10T18:39:56.178254+0000","last_unstale":"2024-09-10T18:40:20.008378+0000","last_undegraded":"2024-09-10T18:40:20.008378+0000","last_fullsized":"2024-09-10T18:40:20.008378+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:21:25.795694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008228+0000","last_change":"2024-09-10T18:39:58.188976+0000","last_active":"2024-09-10T18:40:20.008228+0000","last_peered":"2024-09-10T18:40:20.008228+0000","last_clean":"2024-09-10T18:40:20.008228+0000","last_became_active":"2024-09-10T18:39:58.188679+0000","last_became_peered":"2024-09-10T18:39:58.188679+0000","last_unstale":"2024-09-10T18:40:20.008228+0000","last_undegraded":"2024-09-10T18:40:20.008228+0000","last_fullsized":"2024-09-10T18:40:20.008228+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:37.582865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009655+0000","last_change":"2024-09-10T18:40:00.205242+0000","last_active":"2024-09-10T18:40:20.009655+0000","last_peered":"2024-09-10T18:40:20.009655+0000","last_clean":"2024-09-10T18:40:20.009655+0000","last_became_active":"2024-09-10T18:40:00.204963+0000","last_became_peered":"2024-09-10T18:40:00.204963+0000","last_unstale":"2024-09-10T18:40:20.009655+0000","last_undegraded":"2024-09-10T18:40:20.009655+0000","last_fullsized":"2024-09-10T18:40:20.009655+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:50.342424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008460+0000","last_change":"2024-09-10T18:40:02.218319+0000","last_active":"2024-09-10T18:40:20.008460+0000","last_peered":"2024-09-10T18:40:20.008460+0000","last_clean":"2024-09-10T18:40:20.008460+0000","last_became_active":"2024-09-10T18:40:02.218026+0000","last_became_peered":"2024-09-10T18:40:02.218026+0000","last_unstale":"2024-09-10T18:40:20.008460+0000","last_undegraded":"2024-09-10T18:40:20.008460+0000","last_fullsized":"2024-09-10T18:40:20.008460+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:33.343417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006716+0000","last_change":"2024-09-10T18:39:56.170140+0000","last_active":"2024-09-10T18:40:20.006716+0000","last_peered":"2024-09-10T18:40:20.006716+0000","last_clean":"2024-09-10T18:40:20.006716+0000","last_became_active":"2024-09-10T18:39:56.169876+0000","last_became_peered":"2024-09-10T18:39:56.169876+0000","last_unstale":"2024-09-10T18:40:20.006716+0000","last_undegraded":"2024-09-10T18:40:20.006716+0000","last_fullsized":"2024-09-10T18:40:20.006716+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:15.453448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"67'5","reported_seq":28,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009099+0000","last_change":"2024-09-10T18:39:58.189175+0000","last_active":"2024-09-10T18:40:20.009099+0000","last_peered":"2024-09-10T18:40:20.009099+0000","last_clean":"2024-09-10T18:40:20.009099+0000","last_became_active":"2024-09-10T18:39:58.188879+0000","last_became_peered":"2024-09-10T18:39:58.188879+0000","last_unstale":"2024-09-10T18:40:20.009099+0000","last_undegraded":"2024-09-10T18:40:20.009099+0000","last_fullsized":"2024-09-10T18:40:20.009099+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:10:38.697748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006161+0000","last_change":"2024-09-10T18:40:00.199160+0000","last_active":"2024-09-10T18:40:20.006161+0000","last_peered":"2024-09-10T18:40:20.006161+0000","last_clean":"2024-09-10T18:40:20.006161+0000","last_became_active":"2024-09-10T18:40:00.198905+0000","last_became_peered":"2024-09-10T18:40:00.198905+0000","last_unstale":"2024-09-10T18:40:20.006161+0000","last_undegraded":"2024-09-10T18:40:20.006161+0000","last_fullsized":"2024-09-10T18:40:20.006161+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:39:20.323503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244106+0000","last_change":"2024-09-10T18:40:02.219005+0000","last_active":"2024-09-10T18:40:20.244106+0000","last_peered":"2024-09-10T18:40:20.244106+0000","last_clean":"2024-09-10T18:40:20.244106+0000","last_became_active":"2024-09-10T18:40:02.218275+0000","last_became_peered":"2024-09-10T18:40:02.218275+0000","last_unstale":"2024-09-10T18:40:20.244106+0000","last_undegraded":"2024-09-10T18:40:20.244106+0000","last_fullsized":"2024-09-10T18:40:20.244106+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:57.728322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007209+0000","last_change":"2024-09-10T18:39:56.170759+0000","last_active":"2024-09-10T18:40:20.007209+0000","last_peered":"2024-09-10T18:40:20.007209+0000","last_clean":"2024-09-10T18:40:20.007209+0000","last_became_active":"2024-09-10T18:39:56.170498+0000","last_became_peered":"2024-09-10T18:39:56.170498+0000","last_unstale":"2024-09-10T18:40:20.007209+0000","last_undegraded":"2024-09-10T18:40:20.007209+0000","last_fullsized":"2024-09-10T18:40:20.007209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:32.416607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008706+0000","last_change":"2024-09-10T18:39:58.186155+0000","last_active":"2024-09-10T18:40:20.008706+0000","last_peered":"2024-09-10T18:40:20.008706+0000","last_clean":"2024-09-10T18:40:20.008706+0000","last_became_active":"2024-09-10T18:39:58.185409+0000","last_became_peered":"2024-09-10T18:39:58.185409+0000","last_unstale":"2024-09-10T18:40:20.008706+0000","last_undegraded":"2024-09-10T18:40:20.008706+0000","last_fullsized":"2024-09-10T18:40:20.008706+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:46.982605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244142+0000","last_change":"2024-09-10T18:40:00.200041+0000","last_active":"2024-09-10T18:40:20.244142+0000","last_peered":"2024-09-10T18:40:20.244142+0000","last_clean":"2024-09-10T18:40:20.244142+0000","last_became_active":"2024-09-10T18:40:00.199758+0000","last_became_peered":"2024-09-10T18:40:00.199758+0000","last_unstale":"2024-09-10T18:40:20.244142+0000","last_undegraded":"2024-09-10T18:40:20.244142+0000","last_fullsized":"2024-09-10T18:40:20.244142+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:24.375347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243497+0000","last_change":"2024-09-10T18:40:02.218941+0000","last_active":"2024-09-10T18:40:20.243497+0000","last_peered":"2024-09-10T18:40:20.243497+0000","last_clean":"2024-09-10T18:40:20.243497+0000","last_became_active":"2024-09-10T18:40:02.218680+0000","last_became_peered":"2024-09-10T18:40:02.218680+0000","last_unstale":"2024-09-10T18:40:20.243497+0000","last_undegraded":"2024-09-10T18:40:20.243497+0000","last_fullsized":"2024-09-10T18:40:20.243497+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:29:49.826784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"67'17","reported_seq":74,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.254933+0000","last_change":"2024-09-10T18:40:20.254564+0000","last_active":"2024-09-10T18:40:20.254933+0000","last_peered":"2024-09-10T18:40:20.254933+0000","last_clean":"2024-09-10T18:40:20.254933+0000","last_became_active":"2024-09-10T18:40:20.245552+0000","last_became_peered":"2024-09-10T18:40:20.245552+0000","last_unstale":"2024-09-10T18:40:20.254933+0000","last_undegraded":"2024-09-10T18:40:20.254933+0000","last_fullsized":"2024-09-10T18:40:20.254933+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:58:51.213000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006709+0000","last_change":"2024-09-10T18:39:56.168767+0000","last_active":"2024-09-10T18:40:20.006709+0000","last_peered":"2024-09-10T18:40:20.006709+0000","last_clean":"2024-09-10T18:40:20.006709+0000","last_became_active":"2024-09-10T18:39:56.168515+0000","last_became_peered":"2024-09-10T18:39:56.168515+0000","last_unstale":"2024-09-10T18:40:20.006709+0000","last_undegraded":"2024-09-10T18:40:20.006709+0000","last_fullsized":"2024-09-10T18:40:20.006709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:50.679751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011957+0000","last_change":"2024-09-10T18:40:00.199924+0000","last_active":"2024-09-10T18:40:20.011957+0000","last_peered":"2024-09-10T18:40:20.011957+0000","last_clean":"2024-09-10T18:40:20.011957+0000","last_became_active":"2024-09-10T18:40:00.199629+0000","last_became_peered":"2024-09-10T18:40:00.199629+0000","last_unstale":"2024-09-10T18:40:20.011957+0000","last_undegraded":"2024-09-10T18:40:20.011957+0000","last_fullsized":"2024-09-10T18:40:20.011957+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:33:36.589734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006193+0000","last_change":"2024-09-10T18:40:02.218791+0000","last_active":"2024-09-10T18:40:20.006193+0000","last_peered":"2024-09-10T18:40:20.006193+0000","last_clean":"2024-09-10T18:40:20.006193+0000","last_became_active":"2024-09-10T18:40:02.218530+0000","last_became_peered":"2024-09-10T18:40:02.218530+0000","last_unstale":"2024-09-10T18:40:20.006193+0000","last_undegraded":"2024-09-10T18:40:20.006193+0000","last_fullsized":"2024-09-10T18:40:20.006193+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:57:33.515989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009270+0000","last_change":"2024-09-10T18:39:58.188803+0000","last_active":"2024-09-10T18:40:20.009270+0000","last_peered":"2024-09-10T18:40:20.009270+0000","last_clean":"2024-09-10T18:40:20.009270+0000","last_became_active":"2024-09-10T18:39:58.188075+0000","last_became_peered":"2024-09-10T18:39:58.188075+0000","last_unstale":"2024-09-10T18:40:20.009270+0000","last_undegraded":"2024-09-10T18:40:20.009270+0000","last_fullsized":"2024-09-10T18:40:20.009270+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:57.195997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011212+0000","last_change":"2024-09-10T18:39:56.175101+0000","last_active":"2024-09-10T18:40:20.011212+0000","last_peered":"2024-09-10T18:40:20.011212+0000","last_clean":"2024-09-10T18:40:20.011212+0000","last_became_active":"2024-09-10T18:39:56.173634+0000","last_became_peered":"2024-09-10T18:39:56.173634+0000","last_unstale":"2024-09-10T18:40:20.011212+0000","last_undegraded":"2024-09-10T18:40:20.011212+0000","last_fullsized":"2024-09-10T18:40:20.011212+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:59.655949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294880+0000","last_change":"2024-09-10T18:40:00.203020+0000","last_active":"2024-09-10T18:41:19.294880+0000","last_peered":"2024-09-10T18:41:19.294880+0000","last_clean":"2024-09-10T18:41:19.294880+0000","last_became_active":"2024-09-10T18:40:00.202719+0000","last_became_peered":"2024-09-10T18:40:00.202719+0000","last_unstale":"2024-09-10T18:41:19.294880+0000","last_undegraded":"2024-09-10T18:41:19.294880+0000","last_fullsized":"2024-09-10T18:41:19.294880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:03.397133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008706+0000","last_change":"2024-09-10T18:40:02.213562+0000","last_active":"2024-09-10T18:40:20.008706+0000","last_peered":"2024-09-10T18:40:20.008706+0000","last_clean":"2024-09-10T18:40:20.008706+0000","last_became_active":"2024-09-10T18:40:02.213308+0000","last_became_peered":"2024-09-10T18:40:02.213308+0000","last_unstale":"2024-09-10T18:40:20.008706+0000","last_undegraded":"2024-09-10T18:40:20.008706+0000","last_fullsized":"2024-09-10T18:40:20.008706+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.690924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011858+0000","last_change":"2024-09-10T18:39:58.189590+0000","last_active":"2024-09-10T18:40:20.011858+0000","last_peered":"2024-09-10T18:40:20.011858+0000","last_clean":"2024-09-10T18:40:20.011858+0000","last_became_active":"2024-09-10T18:39:58.189261+0000","last_became_peered":"2024-09-10T18:39:58.189261+0000","last_unstale":"2024-09-10T18:40:20.011858+0000","last_undegraded":"2024-09-10T18:40:20.011858+0000","last_fullsized":"2024-09-10T18:40:20.011858+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:00.793497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010524+0000","last_change":"2024-09-10T18:39:56.176475+0000","last_active":"2024-09-10T18:40:20.010524+0000","last_peered":"2024-09-10T18:40:20.010524+0000","last_clean":"2024-09-10T18:40:20.010524+0000","last_became_active":"2024-09-10T18:39:56.173821+0000","last_became_peered":"2024-09-10T18:39:56.173821+0000","last_unstale":"2024-09-10T18:40:20.010524+0000","last_undegraded":"2024-09-10T18:40:20.010524+0000","last_fullsized":"2024-09-10T18:40:20.010524+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:54.059952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295073+0000","last_change":"2024-09-10T18:40:00.205245+0000","last_active":"2024-09-10T18:41:19.295073+0000","last_peered":"2024-09-10T18:41:19.295073+0000","last_clean":"2024-09-10T18:41:19.295073+0000","last_became_active":"2024-09-10T18:40:00.204887+0000","last_became_peered":"2024-09-10T18:40:00.204887+0000","last_unstale":"2024-09-10T18:41:19.295073+0000","last_undegraded":"2024-09-10T18:41:19.295073+0000","last_fullsized":"2024-09-10T18:41:19.295073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:02:34.589332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244801+0000","last_change":"2024-09-10T18:40:02.217599+0000","last_active":"2024-09-10T18:40:20.244801+0000","last_peered":"2024-09-10T18:40:20.244801+0000","last_clean":"2024-09-10T18:40:20.244801+0000","last_became_active":"2024-09-10T18:40:02.217314+0000","last_became_peered":"2024-09-10T18:40:02.217314+0000","last_unstale":"2024-09-10T18:40:20.244801+0000","last_undegraded":"2024-09-10T18:40:20.244801+0000","last_fullsized":"2024-09-10T18:40:20.244801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:37.108476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008990+0000","last_change":"2024-09-10T18:39:58.188567+0000","last_active":"2024-09-10T18:40:20.008990+0000","last_peered":"2024-09-10T18:40:20.008990+0000","last_clean":"2024-09-10T18:40:20.008990+0000","last_became_active":"2024-09-10T18:39:58.188189+0000","last_became_peered":"2024-09-10T18:39:58.188189+0000","last_unstale":"2024-09-10T18:40:20.008990+0000","last_undegraded":"2024-09-10T18:40:20.008990+0000","last_fullsized":"2024-09-10T18:40:20.008990+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:37.758602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006154+0000","last_change":"2024-09-10T18:39:56.171415+0000","last_active":"2024-09-10T18:40:20.006154+0000","last_peered":"2024-09-10T18:40:20.006154+0000","last_clean":"2024-09-10T18:40:20.006154+0000","last_became_active":"2024-09-10T18:39:56.170065+0000","last_became_peered":"2024-09-10T18:39:56.170065+0000","last_unstale":"2024-09-10T18:40:20.006154+0000","last_undegraded":"2024-09-10T18:40:20.006154+0000","last_fullsized":"2024-09-10T18:40:20.006154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:54.603653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244481+0000","last_change":"2024-09-10T18:40:00.206942+0000","last_active":"2024-09-10T18:40:20.244481+0000","last_peered":"2024-09-10T18:40:20.244481+0000","last_clean":"2024-09-10T18:40:20.244481+0000","last_became_active":"2024-09-10T18:40:00.206630+0000","last_became_peered":"2024-09-10T18:40:00.206630+0000","last_unstale":"2024-09-10T18:40:20.244481+0000","last_undegraded":"2024-09-10T18:40:20.244481+0000","last_fullsized":"2024-09-10T18:40:20.244481+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:12.397846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007438+0000","last_change":"2024-09-10T18:40:02.643603+0000","last_active":"2024-09-10T18:40:20.007438+0000","last_peered":"2024-09-10T18:40:20.007438+0000","last_clean":"2024-09-10T18:40:20.007438+0000","last_became_active":"2024-09-10T18:40:02.643128+0000","last_became_peered":"2024-09-10T18:40:02.643128+0000","last_unstale":"2024-09-10T18:40:20.007438+0000","last_undegraded":"2024-09-10T18:40:20.007438+0000","last_fullsized":"2024-09-10T18:40:20.007438+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:49.775594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"67'14","reported_seq":39,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008652+0000","last_change":"2024-09-10T18:39:58.187128+0000","last_active":"2024-09-10T18:40:20.008652+0000","last_peered":"2024-09-10T18:40:20.008652+0000","last_clean":"2024-09-10T18:40:20.008652+0000","last_became_active":"2024-09-10T18:39:58.186677+0000","last_became_peered":"2024-09-10T18:39:58.186677+0000","last_unstale":"2024-09-10T18:40:20.008652+0000","last_undegraded":"2024-09-10T18:40:20.008652+0000","last_fullsized":"2024-09-10T18:40:20.008652+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:26.652612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008695+0000","last_change":"2024-09-10T18:39:56.168526+0000","last_active":"2024-09-10T18:40:20.008695+0000","last_peered":"2024-09-10T18:40:20.008695+0000","last_clean":"2024-09-10T18:40:20.008695+0000","last_became_active":"2024-09-10T18:39:56.168258+0000","last_became_peered":"2024-09-10T18:39:56.168258+0000","last_unstale":"2024-09-10T18:40:20.008695+0000","last_undegraded":"2024-09-10T18:40:20.008695+0000","last_fullsized":"2024-09-10T18:40:20.008695+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:04:29.952461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294888+0000","last_change":"2024-09-10T18:40:00.202205+0000","last_active":"2024-09-10T18:41:19.294888+0000","last_peered":"2024-09-10T18:41:19.294888+0000","last_clean":"2024-09-10T18:41:19.294888+0000","last_became_active":"2024-09-10T18:40:00.201897+0000","last_became_peered":"2024-09-10T18:40:00.201897+0000","last_unstale":"2024-09-10T18:41:19.294888+0000","last_undegraded":"2024-09-10T18:41:19.294888+0000","last_fullsized":"2024-09-10T18:41:19.294888+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:10.127581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006546+0000","last_change":"2024-09-10T18:40:02.645133+0000","last_active":"2024-09-10T18:40:20.006546+0000","last_peered":"2024-09-10T18:40:20.006546+0000","last_clean":"2024-09-10T18:40:20.006546+0000","last_became_active":"2024-09-10T18:40:02.644840+0000","last_became_peered":"2024-09-10T18:40:02.644840+0000","last_unstale":"2024-09-10T18:40:20.006546+0000","last_undegraded":"2024-09-10T18:40:20.006546+0000","last_fullsized":"2024-09-10T18:40:20.006546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:45.748017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"67'18","reported_seq":50,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006996+0000","last_change":"2024-09-10T18:39:58.185421+0000","last_active":"2024-09-10T18:40:20.006996+0000","last_peered":"2024-09-10T18:40:20.006996+0000","last_clean":"2024-09-10T18:40:20.006996+0000","last_became_active":"2024-09-10T18:39:58.184795+0000","last_became_peered":"2024-09-10T18:39:58.184795+0000","last_unstale":"2024-09-10T18:40:20.006996+0000","last_undegraded":"2024-09-10T18:40:20.006996+0000","last_fullsized":"2024-09-10T18:40:20.006996+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:28:27.224265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"60'1","reported_seq":31,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011583+0000","last_change":"2024-09-10T18:39:56.170415+0000","last_active":"2024-09-10T18:40:20.011583+0000","last_peered":"2024-09-10T18:40:20.011583+0000","last_clean":"2024-09-10T18:40:20.011583+0000","last_became_active":"2024-09-10T18:39:56.170127+0000","last_became_peered":"2024-09-10T18:39:56.170127+0000","last_unstale":"2024-09-10T18:40:20.011583+0000","last_undegraded":"2024-09-10T18:40:20.011583+0000","last_fullsized":"2024-09-10T18:40:20.011583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:50:45.874362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008499+0000","last_change":"2024-09-10T18:40:00.198481+0000","last_active":"2024-09-10T18:40:20.008499+0000","last_peered":"2024-09-10T18:40:20.008499+0000","last_clean":"2024-09-10T18:40:20.008499+0000","last_became_active":"2024-09-10T18:40:00.198066+0000","last_became_peered":"2024-09-10T18:40:00.198066+0000","last_unstale":"2024-09-10T18:40:20.008499+0000","last_undegraded":"2024-09-10T18:40:20.008499+0000","last_fullsized":"2024-09-10T18:40:20.008499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:11.422250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008629+0000","last_change":"2024-09-10T18:40:02.214195+0000","last_active":"2024-09-10T18:40:20.008629+0000","last_peered":"2024-09-10T18:40:20.008629+0000","last_clean":"2024-09-10T18:40:20.008629+0000","last_became_active":"2024-09-10T18:40:02.213936+0000","last_became_peered":"2024-09-10T18:40:02.213936+0000","last_unstale":"2024-09-10T18:40:20.008629+0000","last_undegraded":"2024-09-10T18:40:20.008629+0000","last_fullsized":"2024-09-10T18:40:20.008629+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:06.505320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"67'19","reported_seq":54,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244141+0000","last_change":"2024-09-10T18:39:58.183826+0000","last_active":"2024-09-10T18:40:20.244141+0000","last_peered":"2024-09-10T18:40:20.244141+0000","last_clean":"2024-09-10T18:40:20.244141+0000","last_became_active":"2024-09-10T18:39:58.183542+0000","last_became_peered":"2024-09-10T18:39:58.183542+0000","last_unstale":"2024-09-10T18:40:20.244141+0000","last_undegraded":"2024-09-10T18:40:20.244141+0000","last_fullsized":"2024-09-10T18:40:20.244141+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:02.366162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006760+0000","last_change":"2024-09-10T18:39:56.169355+0000","last_active":"2024-09-10T18:40:20.006760+0000","last_peered":"2024-09-10T18:40:20.006760+0000","last_clean":"2024-09-10T18:40:20.006760+0000","last_became_active":"2024-09-10T18:39:56.169102+0000","last_became_peered":"2024-09-10T18:39:56.169102+0000","last_unstale":"2024-09-10T18:40:20.006760+0000","last_undegraded":"2024-09-10T18:40:20.006760+0000","last_fullsized":"2024-09-10T18:40:20.006760+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:51:23.264894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008348+0000","last_change":"2024-09-10T18:40:00.201031+0000","last_active":"2024-09-10T18:40:20.008348+0000","last_peered":"2024-09-10T18:40:20.008348+0000","last_clean":"2024-09-10T18:40:20.008348+0000","last_became_active":"2024-09-10T18:40:00.200703+0000","last_became_peered":"2024-09-10T18:40:00.200703+0000","last_unstale":"2024-09-10T18:40:20.008348+0000","last_undegraded":"2024-09-10T18:40:20.008348+0000","last_fullsized":"2024-09-10T18:40:20.008348+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:58.060838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009546+0000","last_change":"2024-09-10T18:40:02.644996+0000","last_active":"2024-09-10T18:40:20.009546+0000","last_peered":"2024-09-10T18:40:20.009546+0000","last_clean":"2024-09-10T18:40:20.009546+0000","last_became_active":"2024-09-10T18:40:02.644619+0000","last_became_peered":"2024-09-10T18:40:02.644619+0000","last_unstale":"2024-09-10T18:40:20.009546+0000","last_undegraded":"2024-09-10T18:40:20.009546+0000","last_fullsized":"2024-09-10T18:40:20.009546+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:07.209586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011077+0000","last_change":"2024-09-10T18:39:58.187849+0000","last_active":"2024-09-10T18:40:20.011077+0000","last_peered":"2024-09-10T18:40:20.011077+0000","last_clean":"2024-09-10T18:40:20.011077+0000","last_became_active":"2024-09-10T18:39:58.187225+0000","last_became_peered":"2024-09-10T18:39:58.187225+0000","last_unstale":"2024-09-10T18:40:20.011077+0000","last_undegraded":"2024-09-10T18:40:20.011077+0000","last_fullsized":"2024-09-10T18:40:20.011077+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:52.072943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.012098+0000","last_change":"2024-09-10T18:39:56.173477+0000","last_active":"2024-09-10T18:40:20.012098+0000","last_peered":"2024-09-10T18:40:20.012098+0000","last_clean":"2024-09-10T18:40:20.012098+0000","last_became_active":"2024-09-10T18:39:56.173001+0000","last_became_peered":"2024-09-10T18:39:56.173001+0000","last_unstale":"2024-09-10T18:40:20.012098+0000","last_undegraded":"2024-09-10T18:40:20.012098+0000","last_fullsized":"2024-09-10T18:40:20.012098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:59:46.940431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294880+0000","last_change":"2024-09-10T18:40:00.201429+0000","last_active":"2024-09-10T18:41:19.294880+0000","last_peered":"2024-09-10T18:41:19.294880+0000","last_clean":"2024-09-10T18:41:19.294880+0000","last_became_active":"2024-09-10T18:40:00.201114+0000","last_became_peered":"2024-09-10T18:40:00.201114+0000","last_unstale":"2024-09-10T18:41:19.294880+0000","last_undegraded":"2024-09-10T18:41:19.294880+0000","last_fullsized":"2024-09-10T18:41:19.294880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:27:13.232098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"67'1","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243319+0000","last_change":"2024-09-10T18:40:02.222334+0000","last_active":"2024-09-10T18:40:20.243319+0000","last_peered":"2024-09-10T18:40:20.243319+0000","last_clean":"2024-09-10T18:40:20.243319+0000","last_became_active":"2024-09-10T18:40:02.222044+0000","last_became_peered":"2024-09-10T18:40:02.222044+0000","last_unstale":"2024-09-10T18:40:20.243319+0000","last_undegraded":"2024-09-10T18:40:20.243319+0000","last_fullsized":"2024-09-10T18:40:20.243319+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:54:11.906709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"67'30","reported_seq":87,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294905+0000","last_change":"2024-09-10T18:39:58.182811+0000","last_active":"2024-09-10T18:41:19.294905+0000","last_peered":"2024-09-10T18:41:19.294905+0000","last_clean":"2024-09-10T18:41:19.294905+0000","last_became_active":"2024-09-10T18:39:58.182229+0000","last_became_peered":"2024-09-10T18:39:58.182229+0000","last_unstale":"2024-09-10T18:41:19.294905+0000","last_undegraded":"2024-09-10T18:41:19.294905+0000","last_fullsized":"2024-09-10T18:41:19.294905+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:59.638710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243141+0000","last_change":"2024-09-10T18:39:56.168390+0000","last_active":"2024-09-10T18:40:20.243141+0000","last_peered":"2024-09-10T18:40:20.243141+0000","last_clean":"2024-09-10T18:40:20.243141+0000","last_became_active":"2024-09-10T18:39:56.167729+0000","last_became_peered":"2024-09-10T18:39:56.167729+0000","last_unstale":"2024-09-10T18:40:20.243141+0000","last_undegraded":"2024-09-10T18:40:20.243141+0000","last_fullsized":"2024-09-10T18:40:20.243141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:05.076803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244062+0000","last_change":"2024-09-10T18:40:00.204429+0000","last_active":"2024-09-10T18:40:20.244062+0000","last_peered":"2024-09-10T18:40:20.244062+0000","last_clean":"2024-09-10T18:40:20.244062+0000","last_became_active":"2024-09-10T18:40:00.204109+0000","last_became_peered":"2024-09-10T18:40:00.204109+0000","last_unstale":"2024-09-10T18:40:20.244062+0000","last_undegraded":"2024-09-10T18:40:20.244062+0000","last_fullsized":"2024-09-10T18:40:20.244062+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:58:16.127260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007926+0000","last_change":"2024-09-10T18:40:02.215984+0000","last_active":"2024-09-10T18:40:20.007926+0000","last_peered":"2024-09-10T18:40:20.007926+0000","last_clean":"2024-09-10T18:40:20.007926+0000","last_became_active":"2024-09-10T18:40:02.215685+0000","last_became_peered":"2024-09-10T18:40:02.215685+0000","last_unstale":"2024-09-10T18:40:20.007926+0000","last_undegraded":"2024-09-10T18:40:20.007926+0000","last_fullsized":"2024-09-10T18:40:20.007926+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:08:24.849917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"67'13","reported_seq":45,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011985+0000","last_change":"2024-09-10T18:39:58.183557+0000","last_active":"2024-09-10T18:40:20.011985+0000","last_peered":"2024-09-10T18:40:20.011985+0000","last_clean":"2024-09-10T18:40:20.011985+0000","last_became_active":"2024-09-10T18:39:58.183267+0000","last_became_peered":"2024-09-10T18:39:58.183267+0000","last_unstale":"2024-09-10T18:40:20.011985+0000","last_undegraded":"2024-09-10T18:40:20.011985+0000","last_fullsized":"2024-09-10T18:40:20.011985+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:23:04.112290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"68'5","reported_seq":112,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:26.400278+0000","last_change":"2024-09-10T18:40:01.199339+0000","last_active":"2024-09-10T18:41:26.400278+0000","last_peered":"2024-09-10T18:41:26.400278+0000","last_clean":"2024-09-10T18:41:26.400278+0000","last_became_active":"2024-09-10T18:39:56.169389+0000","last_became_peered":"2024-09-10T18:39:56.169389+0000","last_unstale":"2024-09-10T18:41:26.400278+0000","last_undegraded":"2024-09-10T18:41:26.400278+0000","last_fullsized":"2024-09-10T18:41:26.400278+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:54:26.751398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001450086,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":84,"num_read_kb":79,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006871+0000","last_change":"2024-09-10T18:39:56.175630+0000","last_active":"2024-09-10T18:40:20.006871+0000","last_peered":"2024-09-10T18:40:20.006871+0000","last_clean":"2024-09-10T18:40:20.006871+0000","last_became_active":"2024-09-10T18:39:56.175331+0000","last_became_peered":"2024-09-10T18:39:56.175331+0000","last_unstale":"2024-09-10T18:40:20.006871+0000","last_undegraded":"2024-09-10T18:40:20.006871+0000","last_fullsized":"2024-09-10T18:40:20.006871+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:39.721081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006241+0000","last_change":"2024-09-10T18:40:00.198067+0000","last_active":"2024-09-10T18:40:20.006241+0000","last_peered":"2024-09-10T18:40:20.006241+0000","last_clean":"2024-09-10T18:40:20.006241+0000","last_became_active":"2024-09-10T18:40:00.197805+0000","last_became_peered":"2024-09-10T18:40:00.197805+0000","last_unstale":"2024-09-10T18:40:20.006241+0000","last_undegraded":"2024-09-10T18:40:20.006241+0000","last_fullsized":"2024-09-10T18:40:20.006241+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:20.013355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006508+0000","last_change":"2024-09-10T18:40:02.643088+0000","last_active":"2024-09-10T18:40:20.006508+0000","last_peered":"2024-09-10T18:40:20.006508+0000","last_clean":"2024-09-10T18:40:20.006508+0000","last_became_active":"2024-09-10T18:40:02.642801+0000","last_became_peered":"2024-09-10T18:40:02.642801+0000","last_unstale":"2024-09-10T18:40:20.006508+0000","last_undegraded":"2024-09-10T18:40:20.006508+0000","last_fullsized":"2024-09-10T18:40:20.006508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:29:53.511653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"67'12","reported_seq":36,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244063+0000","last_change":"2024-09-10T18:39:58.182753+0000","last_active":"2024-09-10T18:40:20.244063+0000","last_peered":"2024-09-10T18:40:20.244063+0000","last_clean":"2024-09-10T18:40:20.244063+0000","last_became_active":"2024-09-10T18:39:58.182468+0000","last_became_peered":"2024-09-10T18:39:58.182468+0000","last_unstale":"2024-09-10T18:40:20.244063+0000","last_undegraded":"2024-09-10T18:40:20.244063+0000","last_fullsized":"2024-09-10T18:40:20.244063+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:42.690994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"63'1","reported_seq":26,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.242875+0000","last_change":"2024-09-10T18:40:01.197660+0000","last_active":"2024-09-10T18:40:20.242875+0000","last_peered":"2024-09-10T18:40:20.242875+0000","last_clean":"2024-09-10T18:40:20.242875+0000","last_became_active":"2024-09-10T18:39:56.172423+0000","last_became_peered":"2024-09-10T18:39:56.172423+0000","last_unstale":"2024-09-10T18:40:20.242875+0000","last_undegraded":"2024-09-10T18:40:20.242875+0000","last_fullsized":"2024-09-10T18:40:20.242875+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:21.606985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083450500000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006304+0000","last_change":"2024-09-10T18:39:56.170689+0000","last_active":"2024-09-10T18:40:20.006304+0000","last_peered":"2024-09-10T18:40:20.006304+0000","last_clean":"2024-09-10T18:40:20.006304+0000","last_became_active":"2024-09-10T18:39:56.170428+0000","last_became_peered":"2024-09-10T18:39:56.170428+0000","last_unstale":"2024-09-10T18:40:20.006304+0000","last_undegraded":"2024-09-10T18:40:20.006304+0000","last_fullsized":"2024-09-10T18:40:20.006304+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:14:19.461425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244486+0000","last_change":"2024-09-10T18:40:00.203178+0000","last_active":"2024-09-10T18:40:20.244486+0000","last_peered":"2024-09-10T18:40:20.244486+0000","last_clean":"2024-09-10T18:40:20.244486+0000","last_became_active":"2024-09-10T18:40:00.202894+0000","last_became_peered":"2024-09-10T18:40:00.202894+0000","last_unstale":"2024-09-10T18:40:20.244486+0000","last_undegraded":"2024-09-10T18:40:20.244486+0000","last_fullsized":"2024-09-10T18:40:20.244486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:20.997653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011872+0000","last_change":"2024-09-10T18:40:02.218061+0000","last_active":"2024-09-10T18:40:20.011872+0000","last_peered":"2024-09-10T18:40:20.011872+0000","last_clean":"2024-09-10T18:40:20.011872+0000","last_became_active":"2024-09-10T18:40:02.217811+0000","last_became_peered":"2024-09-10T18:40:02.217811+0000","last_unstale":"2024-09-10T18:40:20.011872+0000","last_undegraded":"2024-09-10T18:40:20.011872+0000","last_fullsized":"2024-09-10T18:40:20.011872+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:05.193655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"67'14","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008523+0000","last_change":"2024-09-10T18:39:58.188235+0000","last_active":"2024-09-10T18:40:20.008523+0000","last_peered":"2024-09-10T18:40:20.008523+0000","last_clean":"2024-09-10T18:40:20.008523+0000","last_became_active":"2024-09-10T18:39:58.187971+0000","last_became_peered":"2024-09-10T18:39:58.187971+0000","last_unstale":"2024-09-10T18:40:20.008523+0000","last_undegraded":"2024-09-10T18:40:20.008523+0000","last_fullsized":"2024-09-10T18:40:20.008523+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:40:33.946776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"65'2","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007502+0000","last_change":"2024-09-10T18:40:01.195073+0000","last_active":"2024-09-10T18:40:20.007502+0000","last_peered":"2024-09-10T18:40:20.007502+0000","last_clean":"2024-09-10T18:40:20.007502+0000","last_became_active":"2024-09-10T18:39:56.171622+0000","last_became_peered":"2024-09-10T18:39:56.171622+0000","last_unstale":"2024-09-10T18:40:20.007502+0000","last_undegraded":"2024-09-10T18:40:20.007502+0000","last_fullsized":"2024-09-10T18:40:20.007502+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:32:21.137605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034161899999999998,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243080+0000","last_change":"2024-09-10T18:39:56.173134+0000","last_active":"2024-09-10T18:40:20.243080+0000","last_peered":"2024-09-10T18:40:20.243080+0000","last_clean":"2024-09-10T18:40:20.243080+0000","last_became_active":"2024-09-10T18:39:56.172764+0000","last_became_peered":"2024-09-10T18:39:56.172764+0000","last_unstale":"2024-09-10T18:40:20.243080+0000","last_undegraded":"2024-09-10T18:40:20.243080+0000","last_fullsized":"2024-09-10T18:40:20.243080+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:37:46.988081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"71'139","reported_seq":273,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:40:38.709571+0000","last_change":"2024-09-10T18:39:47.991205+0000","last_active":"2024-09-10T18:40:38.709571+0000","last_peered":"2024-09-10T18:40:38.709571+0000","last_clean":"2024-09-10T18:40:38.709571+0000","last_became_active":"2024-09-10T18:39:47.990908+0000","last_became_peered":"2024-09-10T18:39:47.990908+0000","last_unstale":"2024-09-10T18:40:38.709571+0000","last_undegraded":"2024-09-10T18:40:38.709571+0000","last_fullsized":"2024-09-10T18:40:38.709571+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:37:39.331017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:37:39.331017+0000","last_clean_scrub_stamp":"2024-09-10T18:37:39.331017+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:58.489157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006486+0000","last_change":"2024-09-10T18:40:00.201459+0000","last_active":"2024-09-10T18:40:20.006486+0000","last_peered":"2024-09-10T18:40:20.006486+0000","last_clean":"2024-09-10T18:40:20.006486+0000","last_became_active":"2024-09-10T18:40:00.201110+0000","last_became_peered":"2024-09-10T18:40:00.201110+0000","last_unstale":"2024-09-10T18:40:20.006486+0000","last_undegraded":"2024-09-10T18:40:20.006486+0000","last_fullsized":"2024-09-10T18:40:20.006486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:20.560962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244562+0000","last_change":"2024-09-10T18:40:02.218501+0000","last_active":"2024-09-10T18:40:20.244562+0000","last_peered":"2024-09-10T18:40:20.244562+0000","last_clean":"2024-09-10T18:40:20.244562+0000","last_became_active":"2024-09-10T18:40:02.218241+0000","last_became_peered":"2024-09-10T18:40:02.218241+0000","last_unstale":"2024-09-10T18:40:20.244562+0000","last_undegraded":"2024-09-10T18:40:20.244562+0000","last_fullsized":"2024-09-10T18:40:20.244562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:26.856011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"67'19","reported_seq":79,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.019754+0000","last_change":"2024-09-10T18:40:20.019451+0000","last_active":"2024-09-10T18:40:20.019754+0000","last_peered":"2024-09-10T18:40:20.019754+0000","last_clean":"2024-09-10T18:40:20.019754+0000","last_became_active":"2024-09-10T18:40:20.010492+0000","last_became_peered":"2024-09-10T18:40:20.010492+0000","last_unstale":"2024-09-10T18:40:20.019754+0000","last_undegraded":"2024-09-10T18:40:20.019754+0000","last_fullsized":"2024-09-10T18:40:20.019754+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:29.006838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006939+0000","last_change":"2024-09-10T18:39:56.168993+0000","last_active":"2024-09-10T18:40:20.006939+0000","last_peered":"2024-09-10T18:40:20.006939+0000","last_clean":"2024-09-10T18:40:20.006939+0000","last_became_active":"2024-09-10T18:39:56.168711+0000","last_became_peered":"2024-09-10T18:39:56.168711+0000","last_unstale":"2024-09-10T18:40:20.006939+0000","last_undegraded":"2024-09-10T18:40:20.006939+0000","last_fullsized":"2024-09-10T18:40:20.006939+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:14.743897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005994+0000","last_change":"2024-09-10T18:40:00.200285+0000","last_active":"2024-09-10T18:40:20.005994+0000","last_peered":"2024-09-10T18:40:20.005994+0000","last_clean":"2024-09-10T18:40:20.005994+0000","last_became_active":"2024-09-10T18:40:00.200000+0000","last_became_peered":"2024-09-10T18:40:00.200000+0000","last_unstale":"2024-09-10T18:40:20.005994+0000","last_undegraded":"2024-09-10T18:40:20.005994+0000","last_fullsized":"2024-09-10T18:40:20.005994+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:27:08.721516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006238+0000","last_change":"2024-09-10T18:40:02.218109+0000","last_active":"2024-09-10T18:40:20.006238+0000","last_peered":"2024-09-10T18:40:20.006238+0000","last_clean":"2024-09-10T18:40:20.006238+0000","last_became_active":"2024-09-10T18:40:02.217802+0000","last_became_peered":"2024-09-10T18:40:02.217802+0000","last_unstale":"2024-09-10T18:40:20.006238+0000","last_undegraded":"2024-09-10T18:40:20.006238+0000","last_fullsized":"2024-09-10T18:40:20.006238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:35.526649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244043+0000","last_change":"2024-09-10T18:39:58.188479+0000","last_active":"2024-09-10T18:40:20.244043+0000","last_peered":"2024-09-10T18:40:20.244043+0000","last_clean":"2024-09-10T18:40:20.244043+0000","last_became_active":"2024-09-10T18:39:58.188185+0000","last_became_peered":"2024-09-10T18:39:58.188185+0000","last_unstale":"2024-09-10T18:40:20.244043+0000","last_undegraded":"2024-09-10T18:40:20.244043+0000","last_fullsized":"2024-09-10T18:40:20.244043+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:38:27.145839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011780+0000","last_change":"2024-09-10T18:39:56.173182+0000","last_active":"2024-09-10T18:40:20.011780+0000","last_peered":"2024-09-10T18:40:20.011780+0000","last_clean":"2024-09-10T18:40:20.011780+0000","last_became_active":"2024-09-10T18:39:56.172836+0000","last_became_peered":"2024-09-10T18:39:56.172836+0000","last_unstale":"2024-09-10T18:40:20.011780+0000","last_undegraded":"2024-09-10T18:40:20.011780+0000","last_fullsized":"2024-09-10T18:40:20.011780+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:35.337251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006252+0000","last_change":"2024-09-10T18:40:00.193028+0000","last_active":"2024-09-10T18:40:20.006252+0000","last_peered":"2024-09-10T18:40:20.006252+0000","last_clean":"2024-09-10T18:40:20.006252+0000","last_became_active":"2024-09-10T18:40:00.192683+0000","last_became_peered":"2024-09-10T18:40:00.192683+0000","last_unstale":"2024-09-10T18:40:20.006252+0000","last_undegraded":"2024-09-10T18:40:20.006252+0000","last_fullsized":"2024-09-10T18:40:20.006252+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:27:55.254130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243852+0000","last_change":"2024-09-10T18:40:02.218043+0000","last_active":"2024-09-10T18:40:20.243852+0000","last_peered":"2024-09-10T18:40:20.243852+0000","last_clean":"2024-09-10T18:40:20.243852+0000","last_became_active":"2024-09-10T18:40:02.217763+0000","last_became_peered":"2024-09-10T18:40:02.217763+0000","last_unstale":"2024-09-10T18:40:20.243852+0000","last_undegraded":"2024-09-10T18:40:20.243852+0000","last_fullsized":"2024-09-10T18:40:20.243852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:27.263161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243975+0000","last_change":"2024-09-10T18:39:58.186251+0000","last_active":"2024-09-10T18:40:20.243975+0000","last_peered":"2024-09-10T18:40:20.243975+0000","last_clean":"2024-09-10T18:40:20.243975+0000","last_became_active":"2024-09-10T18:39:58.185982+0000","last_became_peered":"2024-09-10T18:39:58.185982+0000","last_unstale":"2024-09-10T18:40:20.243975+0000","last_undegraded":"2024-09-10T18:40:20.243975+0000","last_fullsized":"2024-09-10T18:40:20.243975+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:23.438037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":30,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011078+0000","last_change":"2024-09-10T18:40:20.010397+0000","last_active":"2024-09-10T18:40:20.011078+0000","last_peered":"2024-09-10T18:40:20.011078+0000","last_clean":"2024-09-10T18:40:20.011078+0000","last_became_active":"2024-09-10T18:40:20.010069+0000","last_became_peered":"2024-09-10T18:40:20.010069+0000","last_unstale":"2024-09-10T18:40:20.011078+0000","last_undegraded":"2024-09-10T18:40:20.011078+0000","last_fullsized":"2024-09-10T18:40:20.011078+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:05.026738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295102+0000","last_change":"2024-09-10T18:40:00.202488+0000","last_active":"2024-09-10T18:41:19.295102+0000","last_peered":"2024-09-10T18:41:19.295102+0000","last_clean":"2024-09-10T18:41:19.295102+0000","last_became_active":"2024-09-10T18:40:00.202190+0000","last_became_peered":"2024-09-10T18:40:00.202190+0000","last_unstale":"2024-09-10T18:41:19.295102+0000","last_undegraded":"2024-09-10T18:41:19.295102+0000","last_fullsized":"2024-09-10T18:41:19.295102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:52:34.248722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244085+0000","last_change":"2024-09-10T18:40:02.643228+0000","last_active":"2024-09-10T18:40:20.244085+0000","last_peered":"2024-09-10T18:40:20.244085+0000","last_clean":"2024-09-10T18:40:20.244085+0000","last_became_active":"2024-09-10T18:40:02.642890+0000","last_became_peered":"2024-09-10T18:40:02.642890+0000","last_unstale":"2024-09-10T18:40:20.244085+0000","last_undegraded":"2024-09-10T18:40:20.244085+0000","last_fullsized":"2024-09-10T18:40:20.244085+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:43:40.705477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008575+0000","last_change":"2024-09-10T18:39:58.189049+0000","last_active":"2024-09-10T18:40:20.008575+0000","last_peered":"2024-09-10T18:40:20.008575+0000","last_clean":"2024-09-10T18:40:20.008575+0000","last_became_active":"2024-09-10T18:39:58.187544+0000","last_became_peered":"2024-09-10T18:39:58.187544+0000","last_unstale":"2024-09-10T18:40:20.008575+0000","last_undegraded":"2024-09-10T18:40:20.008575+0000","last_fullsized":"2024-09-10T18:40:20.008575+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:45.075380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"60'2","reported_seq":42,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009517+0000","last_change":"2024-09-10T18:39:56.169695+0000","last_active":"2024-09-10T18:40:20.009517+0000","last_peered":"2024-09-10T18:40:20.009517+0000","last_clean":"2024-09-10T18:40:20.009517+0000","last_became_active":"2024-09-10T18:39:56.169408+0000","last_became_peered":"2024-09-10T18:39:56.169408+0000","last_unstale":"2024-09-10T18:40:20.009517+0000","last_undegraded":"2024-09-10T18:40:20.009517+0000","last_fullsized":"2024-09-10T18:40:20.009517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:52.835931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005911+0000","last_change":"2024-09-10T18:40:00.197321+0000","last_active":"2024-09-10T18:40:20.005911+0000","last_peered":"2024-09-10T18:40:20.005911+0000","last_clean":"2024-09-10T18:40:20.005911+0000","last_became_active":"2024-09-10T18:40:00.197025+0000","last_became_peered":"2024-09-10T18:40:00.197025+0000","last_unstale":"2024-09-10T18:40:20.005911+0000","last_undegraded":"2024-09-10T18:40:20.005911+0000","last_fullsized":"2024-09-10T18:40:20.005911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:46.190746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007524+0000","last_change":"2024-09-10T18:40:02.215551+0000","last_active":"2024-09-10T18:40:20.007524+0000","last_peered":"2024-09-10T18:40:20.007524+0000","last_clean":"2024-09-10T18:40:20.007524+0000","last_became_active":"2024-09-10T18:40:02.215223+0000","last_became_peered":"2024-09-10T18:40:02.215223+0000","last_unstale":"2024-09-10T18:40:20.007524+0000","last_undegraded":"2024-09-10T18:40:20.007524+0000","last_fullsized":"2024-09-10T18:40:20.007524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:18.424945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244029+0000","last_change":"2024-09-10T18:39:58.184914+0000","last_active":"2024-09-10T18:40:20.244029+0000","last_peered":"2024-09-10T18:40:20.244029+0000","last_clean":"2024-09-10T18:40:20.244029+0000","last_became_active":"2024-09-10T18:39:58.184624+0000","last_became_peered":"2024-09-10T18:39:58.184624+0000","last_unstale":"2024-09-10T18:40:20.244029+0000","last_undegraded":"2024-09-10T18:40:20.244029+0000","last_fullsized":"2024-09-10T18:40:20.244029+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:42.937378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005970+0000","last_change":"2024-09-10T18:39:56.175019+0000","last_active":"2024-09-10T18:40:20.005970+0000","last_peered":"2024-09-10T18:40:20.005970+0000","last_clean":"2024-09-10T18:40:20.005970+0000","last_became_active":"2024-09-10T18:39:56.174685+0000","last_became_peered":"2024-09-10T18:39:56.174685+0000","last_unstale":"2024-09-10T18:40:20.005970+0000","last_undegraded":"2024-09-10T18:40:20.005970+0000","last_fullsized":"2024-09-10T18:40:20.005970+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:17:07.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008092+0000","last_change":"2024-09-10T18:40:00.198765+0000","last_active":"2024-09-10T18:40:20.008092+0000","last_peered":"2024-09-10T18:40:20.008092+0000","last_clean":"2024-09-10T18:40:20.008092+0000","last_became_active":"2024-09-10T18:40:00.198498+0000","last_became_peered":"2024-09-10T18:40:00.198498+0000","last_unstale":"2024-09-10T18:40:20.008092+0000","last_undegraded":"2024-09-10T18:40:20.008092+0000","last_fullsized":"2024-09-10T18:40:20.008092+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:57.276713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006374+0000","last_change":"2024-09-10T18:40:02.221139+0000","last_active":"2024-09-10T18:40:20.006374+0000","last_peered":"2024-09-10T18:40:20.006374+0000","last_clean":"2024-09-10T18:40:20.006374+0000","last_became_active":"2024-09-10T18:40:02.220858+0000","last_became_peered":"2024-09-10T18:40:02.220858+0000","last_unstale":"2024-09-10T18:40:20.006374+0000","last_undegraded":"2024-09-10T18:40:20.006374+0000","last_fullsized":"2024-09-10T18:40:20.006374+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:27.534137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008228+0000","last_change":"2024-09-10T18:39:56.167572+0000","last_active":"2024-09-10T18:40:20.008228+0000","last_peered":"2024-09-10T18:40:20.008228+0000","last_clean":"2024-09-10T18:40:20.008228+0000","last_became_active":"2024-09-10T18:39:56.167283+0000","last_became_peered":"2024-09-10T18:39:56.167283+0000","last_unstale":"2024-09-10T18:40:20.008228+0000","last_undegraded":"2024-09-10T18:40:20.008228+0000","last_fullsized":"2024-09-10T18:40:20.008228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:04.731029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"67'6","reported_seq":27,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008178+0000","last_change":"2024-09-10T18:39:58.182072+0000","last_active":"2024-09-10T18:40:20.008178+0000","last_peered":"2024-09-10T18:40:20.008178+0000","last_clean":"2024-09-10T18:40:20.008178+0000","last_became_active":"2024-09-10T18:39:58.181824+0000","last_became_peered":"2024-09-10T18:39:58.181824+0000","last_unstale":"2024-09-10T18:40:20.008178+0000","last_undegraded":"2024-09-10T18:40:20.008178+0000","last_fullsized":"2024-09-10T18:40:20.008178+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:30.966225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244472+0000","last_change":"2024-09-10T18:40:00.203124+0000","last_active":"2024-09-10T18:40:20.244472+0000","last_peered":"2024-09-10T18:40:20.244472+0000","last_clean":"2024-09-10T18:40:20.244472+0000","last_became_active":"2024-09-10T18:40:00.202829+0000","last_became_peered":"2024-09-10T18:40:00.202829+0000","last_unstale":"2024-09-10T18:40:20.244472+0000","last_undegraded":"2024-09-10T18:40:20.244472+0000","last_fullsized":"2024-09-10T18:40:20.244472+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:10:12.371398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006455+0000","last_change":"2024-09-10T18:40:02.645417+0000","last_active":"2024-09-10T18:40:20.006455+0000","last_peered":"2024-09-10T18:40:20.006455+0000","last_clean":"2024-09-10T18:40:20.006455+0000","last_became_active":"2024-09-10T18:40:02.645136+0000","last_became_peered":"2024-09-10T18:40:02.645136+0000","last_unstale":"2024-09-10T18:40:20.006455+0000","last_undegraded":"2024-09-10T18:40:20.006455+0000","last_fullsized":"2024-09-10T18:40:20.006455+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:30.247254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007666+0000","last_change":"2024-09-10T18:39:58.182046+0000","last_active":"2024-09-10T18:40:20.007666+0000","last_peered":"2024-09-10T18:40:20.007666+0000","last_clean":"2024-09-10T18:40:20.007666+0000","last_became_active":"2024-09-10T18:39:58.181566+0000","last_became_peered":"2024-09-10T18:39:58.181566+0000","last_unstale":"2024-09-10T18:40:20.007666+0000","last_undegraded":"2024-09-10T18:40:20.007666+0000","last_fullsized":"2024-09-10T18:40:20.007666+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:09:16.291634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243397+0000","last_change":"2024-09-10T18:39:56.173580+0000","last_active":"2024-09-10T18:40:20.243397+0000","last_peered":"2024-09-10T18:40:20.243397+0000","last_clean":"2024-09-10T18:40:20.243397+0000","last_became_active":"2024-09-10T18:39:56.173314+0000","last_became_peered":"2024-09-10T18:39:56.173314+0000","last_unstale":"2024-09-10T18:40:20.243397+0000","last_undegraded":"2024-09-10T18:40:20.243397+0000","last_fullsized":"2024-09-10T18:40:20.243397+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:14.122137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295121+0000","last_change":"2024-09-10T18:40:00.203886+0000","last_active":"2024-09-10T18:41:19.295121+0000","last_peered":"2024-09-10T18:41:19.295121+0000","last_clean":"2024-09-10T18:41:19.295121+0000","last_became_active":"2024-09-10T18:40:00.203601+0000","last_became_peered":"2024-09-10T18:40:00.203601+0000","last_unstale":"2024-09-10T18:41:19.295121+0000","last_undegraded":"2024-09-10T18:41:19.295121+0000","last_fullsized":"2024-09-10T18:41:19.295121+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:03.938604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243877+0000","last_change":"2024-09-10T18:40:02.221141+0000","last_active":"2024-09-10T18:40:20.243877+0000","last_peered":"2024-09-10T18:40:20.243877+0000","last_clean":"2024-09-10T18:40:20.243877+0000","last_became_active":"2024-09-10T18:40:02.220850+0000","last_became_peered":"2024-09-10T18:40:02.220850+0000","last_unstale":"2024-09-10T18:40:20.243877+0000","last_undegraded":"2024-09-10T18:40:20.243877+0000","last_fullsized":"2024-09-10T18:40:20.243877+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:43.215003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.244873+0000","last_change":"2024-09-10T18:39:58.184494+0000","last_active":"2024-09-10T18:40:20.244873+0000","last_peered":"2024-09-10T18:40:20.244873+0000","last_clean":"2024-09-10T18:40:20.244873+0000","last_became_active":"2024-09-10T18:39:58.184244+0000","last_became_peered":"2024-09-10T18:39:58.184244+0000","last_unstale":"2024-09-10T18:40:20.244873+0000","last_undegraded":"2024-09-10T18:40:20.244873+0000","last_fullsized":"2024-09-10T18:40:20.244873+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:40.137215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243202+0000","last_change":"2024-09-10T18:39:56.172917+0000","last_active":"2024-09-10T18:40:20.243202+0000","last_peered":"2024-09-10T18:40:20.243202+0000","last_clean":"2024-09-10T18:40:20.243202+0000","last_became_active":"2024-09-10T18:39:56.172634+0000","last_became_peered":"2024-09-10T18:39:56.172634+0000","last_unstale":"2024-09-10T18:40:20.243202+0000","last_undegraded":"2024-09-10T18:40:20.243202+0000","last_fullsized":"2024-09-10T18:40:20.243202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:01.452156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"67'11","reported_seq":45,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.294929+0000","last_change":"2024-09-10T18:40:00.199630+0000","last_active":"2024-09-10T18:41:19.294929+0000","last_peered":"2024-09-10T18:41:19.294929+0000","last_clean":"2024-09-10T18:41:19.294929+0000","last_became_active":"2024-09-10T18:40:00.199354+0000","last_became_peered":"2024-09-10T18:40:00.199354+0000","last_unstale":"2024-09-10T18:41:19.294929+0000","last_undegraded":"2024-09-10T18:41:19.294929+0000","last_fullsized":"2024-09-10T18:41:19.294929+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:37.611666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009405+0000","last_change":"2024-09-10T18:40:02.217502+0000","last_active":"2024-09-10T18:40:20.009405+0000","last_peered":"2024-09-10T18:40:20.009405+0000","last_clean":"2024-09-10T18:40:20.009405+0000","last_became_active":"2024-09-10T18:40:02.217242+0000","last_became_peered":"2024-09-10T18:40:02.217242+0000","last_unstale":"2024-09-10T18:40:20.009405+0000","last_undegraded":"2024-09-10T18:40:20.009405+0000","last_fullsized":"2024-09-10T18:40:20.009405+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:00:13.097081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"67'9","reported_seq":34,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010980+0000","last_change":"2024-09-10T18:39:58.189204+0000","last_active":"2024-09-10T18:40:20.010980+0000","last_peered":"2024-09-10T18:40:20.010980+0000","last_clean":"2024-09-10T18:40:20.010980+0000","last_became_active":"2024-09-10T18:39:58.188532+0000","last_became_peered":"2024-09-10T18:39:58.188532+0000","last_unstale":"2024-09-10T18:40:20.010980+0000","last_undegraded":"2024-09-10T18:40:20.010980+0000","last_fullsized":"2024-09-10T18:40:20.010980+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:50:31.999746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008327+0000","last_change":"2024-09-10T18:39:56.174627+0000","last_active":"2024-09-10T18:40:20.008327+0000","last_peered":"2024-09-10T18:40:20.008327+0000","last_clean":"2024-09-10T18:40:20.008327+0000","last_became_active":"2024-09-10T18:39:56.174369+0000","last_became_peered":"2024-09-10T18:39:56.174369+0000","last_unstale":"2024-09-10T18:40:20.008327+0000","last_undegraded":"2024-09-10T18:40:20.008327+0000","last_fullsized":"2024-09-10T18:40:20.008327+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:19.919451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007595+0000","last_change":"2024-09-10T18:40:00.203117+0000","last_active":"2024-09-10T18:40:20.007595+0000","last_peered":"2024-09-10T18:40:20.007595+0000","last_clean":"2024-09-10T18:40:20.007595+0000","last_became_active":"2024-09-10T18:40:00.202662+0000","last_became_peered":"2024-09-10T18:40:00.202662+0000","last_unstale":"2024-09-10T18:40:20.007595+0000","last_undegraded":"2024-09-10T18:40:20.007595+0000","last_fullsized":"2024-09-10T18:40:20.007595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:40:08.271516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243290+0000","last_change":"2024-09-10T18:40:02.221280+0000","last_active":"2024-09-10T18:40:20.243290+0000","last_peered":"2024-09-10T18:40:20.243290+0000","last_clean":"2024-09-10T18:40:20.243290+0000","last_became_active":"2024-09-10T18:40:02.221037+0000","last_became_peered":"2024-09-10T18:40:02.221037+0000","last_unstale":"2024-09-10T18:40:20.243290+0000","last_undegraded":"2024-09-10T18:40:20.243290+0000","last_fullsized":"2024-09-10T18:40:20.243290+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.997975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009182+0000","last_change":"2024-09-10T18:39:58.189088+0000","last_active":"2024-09-10T18:40:20.009182+0000","last_peered":"2024-09-10T18:40:20.009182+0000","last_clean":"2024-09-10T18:40:20.009182+0000","last_became_active":"2024-09-10T18:39:58.188846+0000","last_became_peered":"2024-09-10T18:39:58.188846+0000","last_unstale":"2024-09-10T18:40:20.009182+0000","last_undegraded":"2024-09-10T18:40:20.009182+0000","last_fullsized":"2024-09-10T18:40:20.009182+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:19.568943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011709+0000","last_change":"2024-09-10T18:39:56.168804+0000","last_active":"2024-09-10T18:40:20.011709+0000","last_peered":"2024-09-10T18:40:20.011709+0000","last_clean":"2024-09-10T18:40:20.011709+0000","last_became_active":"2024-09-10T18:39:56.168528+0000","last_became_peered":"2024-09-10T18:39:56.168528+0000","last_unstale":"2024-09-10T18:40:20.011709+0000","last_undegraded":"2024-09-10T18:40:20.011709+0000","last_fullsized":"2024-09-10T18:40:20.011709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:45.447696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.010878+0000","last_change":"2024-09-10T18:40:00.199892+0000","last_active":"2024-09-10T18:40:20.010878+0000","last_peered":"2024-09-10T18:40:20.010878+0000","last_clean":"2024-09-10T18:40:20.010878+0000","last_became_active":"2024-09-10T18:40:00.199568+0000","last_became_peered":"2024-09-10T18:40:00.199568+0000","last_unstale":"2024-09-10T18:40:20.010878+0000","last_undegraded":"2024-09-10T18:40:20.010878+0000","last_fullsized":"2024-09-10T18:40:20.010878+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:07.598997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006570+0000","last_change":"2024-09-10T18:40:02.218612+0000","last_active":"2024-09-10T18:40:20.006570+0000","last_peered":"2024-09-10T18:40:20.006570+0000","last_clean":"2024-09-10T18:40:20.006570+0000","last_became_active":"2024-09-10T18:40:02.218310+0000","last_became_peered":"2024-09-10T18:40:02.218310+0000","last_unstale":"2024-09-10T18:40:20.006570+0000","last_undegraded":"2024-09-10T18:40:20.006570+0000","last_fullsized":"2024-09-10T18:40:20.006570+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:43:33.599593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"67'4","reported_seq":40,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.022089+0000","last_change":"2024-09-10T18:40:20.021794+0000","last_active":"2024-09-10T18:40:20.022089+0000","last_peered":"2024-09-10T18:40:20.022089+0000","last_clean":"2024-09-10T18:40:20.022089+0000","last_became_active":"2024-09-10T18:40:20.010027+0000","last_became_peered":"2024-09-10T18:40:20.010027+0000","last_unstale":"2024-09-10T18:40:20.022089+0000","last_undegraded":"2024-09-10T18:40:20.022089+0000","last_fullsized":"2024-09-10T18:40:20.022089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:08.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243127+0000","last_change":"2024-09-10T18:39:56.168677+0000","last_active":"2024-09-10T18:40:20.243127+0000","last_peered":"2024-09-10T18:40:20.243127+0000","last_clean":"2024-09-10T18:40:20.243127+0000","last_became_active":"2024-09-10T18:39:56.168376+0000","last_became_peered":"2024-09-10T18:39:56.168376+0000","last_unstale":"2024-09-10T18:40:20.243127+0000","last_undegraded":"2024-09-10T18:40:20.243127+0000","last_fullsized":"2024-09-10T18:40:20.243127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:46.683638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008468+0000","last_change":"2024-09-10T18:40:00.205229+0000","last_active":"2024-09-10T18:40:20.008468+0000","last_peered":"2024-09-10T18:40:20.008468+0000","last_clean":"2024-09-10T18:40:20.008468+0000","last_became_active":"2024-09-10T18:40:00.204967+0000","last_became_peered":"2024-09-10T18:40:00.204967+0000","last_unstale":"2024-09-10T18:40:20.008468+0000","last_undegraded":"2024-09-10T18:40:20.008468+0000","last_fullsized":"2024-09-10T18:40:20.008468+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:11.921294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006154+0000","last_change":"2024-09-10T18:40:02.216361+0000","last_active":"2024-09-10T18:40:20.006154+0000","last_peered":"2024-09-10T18:40:20.006154+0000","last_clean":"2024-09-10T18:40:20.006154+0000","last_became_active":"2024-09-10T18:40:02.216055+0000","last_became_peered":"2024-09-10T18:40:02.216055+0000","last_unstale":"2024-09-10T18:40:20.006154+0000","last_undegraded":"2024-09-10T18:40:20.006154+0000","last_fullsized":"2024-09-10T18:40:20.006154+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:26.629430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007152+0000","last_change":"2024-09-10T18:39:58.187272+0000","last_active":"2024-09-10T18:40:20.007152+0000","last_peered":"2024-09-10T18:40:20.007152+0000","last_clean":"2024-09-10T18:40:20.007152+0000","last_became_active":"2024-09-10T18:39:58.186202+0000","last_became_peered":"2024-09-10T18:39:58.186202+0000","last_unstale":"2024-09-10T18:40:20.007152+0000","last_undegraded":"2024-09-10T18:40:20.007152+0000","last_fullsized":"2024-09-10T18:40:20.007152+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:19.241984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008635+0000","last_change":"2024-09-10T18:39:56.173502+0000","last_active":"2024-09-10T18:40:20.008635+0000","last_peered":"2024-09-10T18:40:20.008635+0000","last_clean":"2024-09-10T18:40:20.008635+0000","last_became_active":"2024-09-10T18:39:56.173248+0000","last_became_peered":"2024-09-10T18:39:56.173248+0000","last_unstale":"2024-09-10T18:40:20.008635+0000","last_undegraded":"2024-09-10T18:40:20.008635+0000","last_fullsized":"2024-09-10T18:40:20.008635+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:36:28.812714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.005992+0000","last_change":"2024-09-10T18:40:00.202354+0000","last_active":"2024-09-10T18:40:20.005992+0000","last_peered":"2024-09-10T18:40:20.005992+0000","last_clean":"2024-09-10T18:40:20.005992+0000","last_became_active":"2024-09-10T18:40:00.202030+0000","last_became_peered":"2024-09-10T18:40:00.202030+0000","last_unstale":"2024-09-10T18:40:20.005992+0000","last_undegraded":"2024-09-10T18:40:20.005992+0000","last_fullsized":"2024-09-10T18:40:20.005992+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:22.805489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007445+0000","last_change":"2024-09-10T18:40:02.643876+0000","last_active":"2024-09-10T18:40:20.007445+0000","last_peered":"2024-09-10T18:40:20.007445+0000","last_clean":"2024-09-10T18:40:20.007445+0000","last_became_active":"2024-09-10T18:40:02.643249+0000","last_became_peered":"2024-09-10T18:40:02.643249+0000","last_unstale":"2024-09-10T18:40:20.007445+0000","last_undegraded":"2024-09-10T18:40:20.007445+0000","last_fullsized":"2024-09-10T18:40:20.007445+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:22.913203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"67'1","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006417+0000","last_change":"2024-09-10T18:40:02.222541+0000","last_active":"2024-09-10T18:40:20.006417+0000","last_peered":"2024-09-10T18:40:20.006417+0000","last_clean":"2024-09-10T18:40:20.006417+0000","last_became_active":"2024-09-10T18:40:02.222280+0000","last_became_peered":"2024-09-10T18:40:02.222280+0000","last_unstale":"2024-09-10T18:40:20.006417+0000","last_undegraded":"2024-09-10T18:40:20.006417+0000","last_fullsized":"2024-09-10T18:40:20.006417+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:34:40.368982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243061+0000","last_change":"2024-09-10T18:39:56.172788+0000","last_active":"2024-09-10T18:40:20.243061+0000","last_peered":"2024-09-10T18:40:20.243061+0000","last_clean":"2024-09-10T18:40:20.243061+0000","last_became_active":"2024-09-10T18:39:56.172543+0000","last_became_peered":"2024-09-10T18:39:56.172543+0000","last_unstale":"2024-09-10T18:40:20.243061+0000","last_undegraded":"2024-09-10T18:40:20.243061+0000","last_fullsized":"2024-09-10T18:40:20.243061+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:37.211986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"67'10","reported_seq":33,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243977+0000","last_change":"2024-09-10T18:39:58.185397+0000","last_active":"2024-09-10T18:40:20.243977+0000","last_peered":"2024-09-10T18:40:20.243977+0000","last_clean":"2024-09-10T18:40:20.243977+0000","last_became_active":"2024-09-10T18:39:58.185088+0000","last_became_peered":"2024-09-10T18:39:58.185088+0000","last_unstale":"2024-09-10T18:40:20.243977+0000","last_undegraded":"2024-09-10T18:40:20.243977+0000","last_fullsized":"2024-09-10T18:40:20.243977+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:21.988849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"67'11","reported_seq":48,"reported_epoch":71,"state":"active+clean","last_fresh":"2024-09-10T18:41:19.295073+0000","last_change":"2024-09-10T18:40:00.208003+0000","last_active":"2024-09-10T18:41:19.295073+0000","last_peered":"2024-09-10T18:41:19.295073+0000","last_clean":"2024-09-10T18:41:19.295073+0000","last_became_active":"2024-09-10T18:40:00.206620+0000","last_became_peered":"2024-09-10T18:40:00.206620+0000","last_unstale":"2024-09-10T18:41:19.295073+0000","last_undegraded":"2024-09-10T18:41:19.295073+0000","last_fullsized":"2024-09-10T18:41:19.295073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:48.044558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011332+0000","last_change":"2024-09-10T18:40:02.218949+0000","last_active":"2024-09-10T18:40:20.011332+0000","last_peered":"2024-09-10T18:40:20.011332+0000","last_clean":"2024-09-10T18:40:20.011332+0000","last_became_active":"2024-09-10T18:40:02.218557+0000","last_became_peered":"2024-09-10T18:40:02.218557+0000","last_unstale":"2024-09-10T18:40:20.011332+0000","last_undegraded":"2024-09-10T18:40:20.011332+0000","last_fullsized":"2024-09-10T18:40:20.011332+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:14.161856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"60'1","reported_seq":29,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007315+0000","last_change":"2024-09-10T18:39:56.172799+0000","last_active":"2024-09-10T18:40:20.007315+0000","last_peered":"2024-09-10T18:40:20.007315+0000","last_clean":"2024-09-10T18:40:20.007315+0000","last_became_active":"2024-09-10T18:39:56.172533+0000","last_became_peered":"2024-09-10T18:39:56.172533+0000","last_unstale":"2024-09-10T18:40:20.007315+0000","last_undegraded":"2024-09-10T18:40:20.007315+0000","last_fullsized":"2024-09-10T18:40:20.007315+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:24:14.896618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":8,"num_read_kb":8,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"67'11","reported_seq":37,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008615+0000","last_change":"2024-09-10T18:39:58.193514+0000","last_active":"2024-09-10T18:40:20.008615+0000","last_peered":"2024-09-10T18:40:20.008615+0000","last_clean":"2024-09-10T18:40:20.008615+0000","last_became_active":"2024-09-10T18:39:58.193136+0000","last_became_peered":"2024-09-10T18:39:58.193136+0000","last_unstale":"2024-09-10T18:40:20.008615+0000","last_undegraded":"2024-09-10T18:40:20.008615+0000","last_fullsized":"2024-09-10T18:40:20.008615+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:31:26.321586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.243873+0000","last_change":"2024-09-10T18:40:00.200693+0000","last_active":"2024-09-10T18:40:20.243873+0000","last_peered":"2024-09-10T18:40:20.243873+0000","last_clean":"2024-09-10T18:40:20.243873+0000","last_became_active":"2024-09-10T18:40:00.200405+0000","last_became_peered":"2024-09-10T18:40:00.200405+0000","last_unstale":"2024-09-10T18:40:20.243873+0000","last_undegraded":"2024-09-10T18:40:20.243873+0000","last_fullsized":"2024-09-10T18:40:20.243873+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:39.635593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008831+0000","last_change":"2024-09-10T18:40:02.643206+0000","last_active":"2024-09-10T18:40:20.008831+0000","last_peered":"2024-09-10T18:40:20.008831+0000","last_clean":"2024-09-10T18:40:20.008831+0000","last_became_active":"2024-09-10T18:40:02.642935+0000","last_became_peered":"2024-09-10T18:40:02.642935+0000","last_unstale":"2024-09-10T18:40:20.008831+0000","last_undegraded":"2024-09-10T18:40:20.008831+0000","last_fullsized":"2024-09-10T18:40:20.008831+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:25.174406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008811+0000","last_change":"2024-09-10T18:39:56.170131+0000","last_active":"2024-09-10T18:40:20.008811+0000","last_peered":"2024-09-10T18:40:20.008811+0000","last_clean":"2024-09-10T18:40:20.008811+0000","last_became_active":"2024-09-10T18:39:56.169869+0000","last_became_peered":"2024-09-10T18:39:56.169869+0000","last_unstale":"2024-09-10T18:40:20.008811+0000","last_undegraded":"2024-09-10T18:40:20.008811+0000","last_fullsized":"2024-09-10T18:40:20.008811+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:28.042823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"67'15","reported_seq":43,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.006687+0000","last_change":"2024-09-10T18:39:58.179822+0000","last_active":"2024-09-10T18:40:20.006687+0000","last_peered":"2024-09-10T18:40:20.006687+0000","last_clean":"2024-09-10T18:40:20.006687+0000","last_became_active":"2024-09-10T18:39:58.179530+0000","last_became_peered":"2024-09-10T18:39:58.179530+0000","last_unstale":"2024-09-10T18:40:20.006687+0000","last_undegraded":"2024-09-10T18:40:20.006687+0000","last_fullsized":"2024-09-10T18:40:20.006687+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:39.742732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.011571+0000","last_change":"2024-09-10T18:40:00.201621+0000","last_active":"2024-09-10T18:40:20.011571+0000","last_peered":"2024-09-10T18:40:20.011571+0000","last_clean":"2024-09-10T18:40:20.011571+0000","last_became_active":"2024-09-10T18:40:00.201264+0000","last_became_peered":"2024-09-10T18:40:00.201264+0000","last_unstale":"2024-09-10T18:40:20.011571+0000","last_undegraded":"2024-09-10T18:40:20.011571+0000","last_fullsized":"2024-09-10T18:40:20.011571+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:55.861590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":14,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.008088+0000","last_change":"2024-09-10T18:40:02.643833+0000","last_active":"2024-09-10T18:40:20.008088+0000","last_peered":"2024-09-10T18:40:20.008088+0000","last_clean":"2024-09-10T18:40:20.008088+0000","last_became_active":"2024-09-10T18:40:02.643245+0000","last_became_peered":"2024-09-10T18:40:02.643245+0000","last_unstale":"2024-09-10T18:40:20.008088+0000","last_undegraded":"2024-09-10T18:40:20.008088+0000","last_fullsized":"2024-09-10T18:40:20.008088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:40:01.186957+0000","last_clean_scrub_stamp":"2024-09-10T18:40:01.186957+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:10:53.773097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":20,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007711+0000","last_change":"2024-09-10T18:39:56.178535+0000","last_active":"2024-09-10T18:40:20.007711+0000","last_peered":"2024-09-10T18:40:20.007711+0000","last_clean":"2024-09-10T18:40:20.007711+0000","last_became_active":"2024-09-10T18:39:56.178240+0000","last_became_peered":"2024-09-10T18:39:56.178240+0000","last_unstale":"2024-09-10T18:40:20.007711+0000","last_undegraded":"2024-09-10T18:40:20.007711+0000","last_fullsized":"2024-09-10T18:40:20.007711+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:55.142880+0000","last_clean_scrub_stamp":"2024-09-10T18:39:55.142880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:01:01.394833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"67'12","reported_seq":41,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.007824+0000","last_change":"2024-09-10T18:39:58.187539+0000","last_active":"2024-09-10T18:40:20.007824+0000","last_peered":"2024-09-10T18:40:20.007824+0000","last_clean":"2024-09-10T18:40:20.007824+0000","last_became_active":"2024-09-10T18:39:58.187231+0000","last_became_peered":"2024-09-10T18:39:58.187231+0000","last_unstale":"2024-09-10T18:40:20.007824+0000","last_undegraded":"2024-09-10T18:40:20.007824+0000","last_fullsized":"2024-09-10T18:40:20.007824+0000","mapping_epoch":61,"log_start":"0'0","ondisk_log_start":"0'0","created":61,"last_epoch_clean":62,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:57.160039+0000","last_clean_scrub_stamp":"2024-09-10T18:39:57.160039+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:43.864399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T18:40:20.009194+0000","last_change":"2024-09-10T18:40:00.199135+0000","last_active":"2024-09-10T18:40:20.009194+0000","last_peered":"2024-09-10T18:40:20.009194+0000","last_clean":"2024-09-10T18:40:20.009194+0000","last_became_active":"2024-09-10T18:40:00.198871+0000","last_became_peered":"2024-09-10T18:40:00.198871+0000","last_unstale":"2024-09-10T18:40:20.009194+0000","last_undegraded":"2024-09-10T18:40:20.009194+0000","last_fullsized":"2024-09-10T18:40:20.009194+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:39:59.171963+0000","last_clean_scrub_stamp":"2024-09-10T18:39:59.171963+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:38:19.050238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":6,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":84,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":38,"num_read_kb":38,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":156,"num_read_kb":258,"num_write":179,"num_write_kb":3194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":54,"seq":231928234008,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298696,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442360,"statfs":{"total":95990841344,"available":95684976640,"internally_reserved":0,"allocated":1376256,"data_stored":921457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.63700000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.624},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.66300000000000003}]},{"osd":3,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"last":0.68500000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.36399999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.53700000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495643,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298696,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442360,"statfs":{"total":95990841344,"available":95684976640,"internally_reserved":0,"allocated":1372160,"data_stored":922420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.48999999999999999,"15min":0.49199999999999999},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.629},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.53000000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.51000000000000001,"15min":0.51200000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.51100000000000001,"15min":0.51500000000000001},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.47299999999999998,"15min":0.47499999999999998},"min":{"1min":0.36799999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.628,"5min":0.753,"15min":0.753},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51500000000000001,"15min":0.51400000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.443}]},{"osd":3,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.48499999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.58399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.51400000000000001,"15min":0.51800000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.66400000000000003,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.73099999999999998}]},{"osd":4,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.42699999999999999,"15min":0.42899999999999999},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.41799999999999998,"15min":0.41999999999999998},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"last":0.498}]},{"osd":5,"last update":"Tue Sep 10 18:41:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44500000000000001},"min":{"1min":0.30599999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.436,"5min":0.433,"15min":0.433},"min":{"1min":0.32400000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.61199999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.65500000000000003}]}]},{"osd":5,"up_from":38,"seq":163208757281,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298112,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442944,"statfs":{"total":95990841344,"available":95685574656,"internally_reserved":0,"allocated":774144,"data_stored":330862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.505,"15min":0.51100000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59799999999999998,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.61799999999999999,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64200000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.51100000000000001,"15min":0.51300000000000001},"min":{"1min":0.373,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.66800000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.498,"5min":0.52500000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.64500000000000002,"5min":0.749,"15min":0.749},"last":0.59099999999999997}]},{"osd":2,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.497,"15min":0.502},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66500000000000004,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.53800000000000003},"min":{"1min":0.42299999999999999,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.627,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.60199999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52300000000000002,"15min":0.52500000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.65400000000000003,"5min":0.754,"15min":0.754},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55500000000000005},"min":{"1min":0.437,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.71799999999999997,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.67800000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:41:02 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.40699999999999997,"15min":0.39100000000000001},"min":{"1min":0.34300000000000003,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.41499999999999998,"15min":0.40100000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.68400000000000005,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.44600000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.443,"15min":0.42899999999999999},"min":{"1min":0.34200000000000003,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48899999999999999,"15min":0.47899999999999998},"min":{"1min":0.376,"5min":0.314,"15min":0.314},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.46300000000000002}]}]},{"osd":4,"up_from":30,"seq":128849018918,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298132,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442924,"statfs":{"total":95990841344,"available":95685554176,"internally_reserved":0,"allocated":794624,"data_stored":332137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.97499999999999998,"5min":0.59599999999999997,"15min":0.53300000000000003},"min":{"1min":0.40300000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":9.4949999999999992,"5min":9.4949999999999992,"15min":9.4949999999999992},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47899999999999998,"15min":0.47699999999999998},"min":{"1min":0.36199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.30199999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.52200000000000002,"15min":0.52300000000000002},"min":{"1min":0.34399999999999997,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.65900000000000003,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.97599999999999998,"5min":0.57299999999999995,"15min":0.50600000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":9.5389999999999997,"5min":9.5389999999999997,"15min":9.5389999999999997},"last":0.45000000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.51400000000000001,"15min":0.51600000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.71899999999999997,"5min":0.746,"15min":0.746},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.49099999999999999},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.68999999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.435}]},{"osd":3,"last update":"Tue Sep 10 18:40:31 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.60899999999999999,"15min":0.54600000000000004},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":9.5199999999999996,"5min":9.5199999999999996,"15min":9.5199999999999996},"last":0.52500000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:40:56 2024","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.35399999999999998,"15min":0.34399999999999997},"min":{"1min":0.252,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.68300000000000005,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.36899999999999999,"15min":0.36099999999999999},"min":{"1min":0.26600000000000001,"5min":0.251,"15min":0.251},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.50800000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:22 2024","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.41199999999999998,"15min":0.40200000000000002},"min":{"1min":0.31900000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.41899999999999998,"15min":0.40799999999999997},"min":{"1min":0.33900000000000002,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.56599999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:40:47 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59399999999999997}]}]},{"osd":3,"up_from":25,"seq":107374182442,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442904,"statfs":{"total":95990841344,"available":95685533696,"internally_reserved":0,"allocated":815104,"data_stored":331989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.23400000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.54900000000000004,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39000000000000001,"15min":0.38700000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"last":0.317}]},{"osd":1,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.379,"15min":0.379},"min":{"1min":0.28199999999999997,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.27300000000000002,"5min":0.217,"15min":0.217},"max":{"1min":0.58499999999999996,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:41:15 2024","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41499999999999998},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.40500000000000003,"15min":0.40100000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.622,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:40:35 2024","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54400000000000004,"15min":0.54200000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"last":0.441},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.53900000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.374}]},{"osd":5,"last update":"Tue Sep 10 18:40:57 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.58699999999999997,"15min":0.58999999999999997},"min":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.79600000000000004,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.56399999999999995,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.41199999999999998}]},{"osd":7,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.51400000000000001}]}]},{"osd":2,"up_from":18,"seq":77309411376,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298108,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442948,"statfs":{"total":95990841344,"available":95685578752,"internally_reserved":0,"allocated":770048,"data_stored":330450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.372,"15min":0.35999999999999999},"min":{"1min":0.28000000000000003,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.53200000000000003,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.371,"15min":0.36199999999999999},"min":{"1min":0.249,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54600000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.44600000000000001}]},{"osd":1,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38700000000000001,"15min":0.371},"min":{"1min":0.254,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.59299999999999997,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.379,"15min":0.36899999999999999},"min":{"1min":0.27800000000000002,"5min":0.249,"15min":0.249},"max":{"1min":0.69699999999999995,"5min":0.751,"15min":0.751},"last":0.46200000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:41:09 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.434,"15min":0.42599999999999999},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63100000000000001,"5min":0.76800000000000002,"15min":0.76800000000000002},"last":0.434},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.436,"15min":0.432},"min":{"1min":0.308,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54300000000000004,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.503}]},{"osd":4,"last update":"Tue Sep 10 18:40:34 2024","interfaces":[{"interface":"back","average":{"1min":1.7,"5min":0.753,"15min":0.59499999999999997},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":18.253,"5min":18.253,"15min":18.253},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.71099999999999997,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.80200000000000005}]},{"osd":5,"last update":"Tue Sep 10 18:40:59 2024","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.55900000000000005,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.58499999999999996,"15min":0.57699999999999996},"min":{"1min":0.48699999999999999,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76200000000000001,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.72699999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:41:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57499999999999996,"15min":0.56899999999999995},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:40:45 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.75900000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574901,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298152,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442904,"statfs":{"total":95990841344,"available":95685533696,"internally_reserved":0,"allocated":815104,"data_stored":332813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.35299999999999998,"15min":0.33300000000000002},"min":{"1min":0.224,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.49299999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.23499999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.371,"15min":0.33900000000000002},"min":{"1min":0.24299999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.58399999999999996,"5min":0.75,"15min":0.75},"last":0.27200000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:40:41 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.39000000000000001,"15min":0.376},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.41599999999999998,"15min":0.39300000000000002},"min":{"1min":0.27700000000000002,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.495}]},{"osd":3,"last update":"Tue Sep 10 18:41:08 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46100000000000002,"15min":0.45700000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.63,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.41499999999999998,"15min":0.40300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61399999999999999,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.51000000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:40:32 2024","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.85199999999999998,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54900000000000004,"15min":0.54700000000000004},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:40:55 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":0.69399999999999995,"15min":0.628},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":10.497999999999999,"5min":10.497999999999999,"15min":10.497999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.874,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:41:20 2024","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.39300000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.74099999999999999,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.441},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.64600000000000002,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.82399999999999995,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.52400000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:40:46 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.46800000000000003}]}]},{"osd":0,"up_from":8,"seq":34359738427,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":298708,"kb_used_data":1356,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93442348,"statfs":{"total":95990841344,"available":95684964352,"internally_reserved":0,"allocated":1388544,"data_stored":921852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.35399999999999998,"5min":0.35199999999999998,"15min":0.32800000000000001},"min":{"1min":0.23400000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.35199999999999998,"15min":0.34000000000000002},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.55900000000000005,"5min":0.61599999999999999,"15min":0.61599999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:40:48 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.55300000000000005,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.41899999999999998,"15min":0.41299999999999998},"min":{"1min":0.29899999999999999,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:41:13 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.44600000000000001,"15min":0.44900000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.626,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"last":0.378}]},{"osd":4,"last update":"Tue Sep 10 18:40:33 2024","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.56399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.73599999999999999,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54900000000000004,"15min":0.55200000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.71499999999999997,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:40:58 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.77600000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.753,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.58899999999999997}]},{"osd":6,"last update":"Tue Sep 10 18:41:19 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.54700000000000004,"15min":0.54800000000000004},"min":{"1min":0.42299999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.747,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.59999999999999998,"15min":0.60899999999999999},"min":{"1min":0.40999999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.70499999999999996,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Sep 10 18:40:44 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.745,"5min":0.745,"15min":0.745},"last":0.61799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":2118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:41:34.733 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:41:34.733 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:41:34.733 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:41:34.733 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:41:35.315 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:41:34 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:41:34] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:36.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:35 smithi117 ceph-mon[27457]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:36.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:35 smithi117 ceph-mon[27457]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:36.066 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:35 smithi117 bash[52193]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:41:36.066 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:35 smithi117 bash[52193]: Writing manifest to image destination 2024-09-10T18:41:36.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:35 smithi066 ceph-mon[31695]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:36.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:35 smithi066 ceph-mon[31695]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:36.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:35 smithi066 ceph-mon[25398]: from='client.14721 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:41:36.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:35 smithi066 ceph-mon[25398]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:36.477 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 podman[52193]: 2024-09-10 18:41:36.117684206 +0000 UTC m=+2.355243320 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 podman[52193]: 2024-09-10 18:41:36.224580363 +0000 UTC m=+2.462139485 container create 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 podman[52193]: 2024-09-10 18:41:36.418069684 +0000 UTC m=+2.655628799 container init 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 podman[52193]: 2024-09-10 18:41:36.421935891 +0000 UTC m=+2.659495007 container start 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.426Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.426Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.426Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:41:36.478 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:41:36.479 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.427Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:41:36.480 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:41:36.481 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b[52267]: ts=2024-09-10T18:41:36.428Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:41:36.482 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 bash[52193]: 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b 2024-09-10T18:41:36.815 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:41:36 smithi117 systemd[1]: Started Ceph node-exporter.b for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:36.903 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.655 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: Reconfiguring daemon alertmanager.a on smithi066 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.656 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: Reconfiguring daemon alertmanager.a on smithi066 2024-09-10T18:41:37.657 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.658 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.658 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:37 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: Metadata not up to date on all hosts. Skipping non agent specs 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:37.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:41:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: Reconfiguring daemon alertmanager.a on smithi066 2024-09-10T18:41:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:37.818 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:37 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.650 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[31695]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.651 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:39 smithi066 ceph-mon[25398]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:39.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:39 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.655 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:39 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.656 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:39 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:39.656 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:39 smithi117 ceph-mon[27457]: pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:39.656 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:40.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:40 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:40.857 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:40 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:40.858 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:40 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:41.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:41 smithi066 ceph-mon[31695]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:41.728 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:41 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:41.728 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:41 smithi066 ceph-mon[25398]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:41.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:41 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:41.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:41 smithi117 ceph-mon[27457]: pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:41.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:41 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:43.246 INFO:teuthology.orchestra.run.smithi066.stdout: 2024-09-10T18:41:43.246 INFO:teuthology.orchestra.run.smithi066.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:41:43.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:43 smithi066 ceph-mon[31695]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:43.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:43 smithi066 ceph-mon[31695]: from='client.? 172.21.15.66:0/636978923' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:41:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:43 smithi066 ceph-mon[25398]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:43.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:43 smithi066 ceph-mon[25398]: from='client.? 172.21.15.66:0/636978923' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:41:44.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:43 smithi117 ceph-mon[27457]: pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:41:44.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:43 smithi117 ceph-mon[27457]: from='client.? 172.21.15.66:0/636978923' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:41:44.093 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:41:44.093 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:41:44.093 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T18:41:44.109 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T18:41:44.110 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-09-10T18:41:44.983 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:41:44 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:41:44] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:41:45.020 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:45.144 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:45 smithi066 systemd[1]: Stopping Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:45.728 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:45 smithi066 podman[60038]: 2024-09-10 18:41:45.470110134 +0000 UTC m=+0.455844178 container died b9fe91e4b9b641786c446d7000b6687a2481322eefa013a5b3d62f76a4a577ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=festive_visvesvaraya, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD) 2024-09-10T18:41:46.174 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[25398]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:46.174 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.174 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.175 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.175 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[31695]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:46.175 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.175 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.175 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:45 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:45 smithi117 ceph-mon[27457]: pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:41:46.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:45 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:45 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.316 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:45 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:46.770 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:46 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[56755]: ts=2024-09-10T18:41:46.612Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:41:46.878 INFO:teuthology.orchestra.run.smithi066.stdout:167 167 2024-09-10T18:41:47.619 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.619 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.619 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[31695]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:47.619 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[25398]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:47.620 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:47 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.621 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T18:41:47.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:47 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:47 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:47 smithi117 ceph-mon[27457]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:47.817 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:47 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:47.978 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:47 smithi066 podman[60208]: 2024-09-10 18:41:47.662726067 +0000 UTC m=+1.171498757 container died e77077d9c1781fd576e875328b9eb86debc250fbbe5603108379ed6ee4b6d374 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:48.353 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:48 smithi066 podman[60208]: 2024-09-10 18:41:48.184570555 +0000 UTC m=+1.693343244 container cleanup e77077d9c1781fd576e875328b9eb86debc250fbbe5603108379ed6ee4b6d374 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:48.353 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:48 smithi066 bash[60208]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:41:48.523 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:48.646 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:48 smithi066 ceph-mon[31695]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-10T18:41:48.646 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:48 smithi066 ceph-mon[25398]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-10T18:41:48.646 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:48 smithi066 podman[60232]: 2024-09-10 18:41:48.352617754 +0000 UTC m=+1.719262412 container remove e77077d9c1781fd576e875328b9eb86debc250fbbe5603108379ed6ee4b6d374 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:48.646 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:48 smithi066 podman[60232]: 2024-09-10 18:41:48.419701954 +0000 UTC m=+1.786346598 volume remove b98c0f09683cf9196e2d9dd48ec9a78890a7d7409c2a7a4b81f193215353824a 2024-09-10T18:41:48.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:48 smithi117 ceph-mon[27457]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-10T18:41:49.228 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:49 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:41:49.228 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:49 smithi066 systemd[1]: Stopped Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:49.228 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:49 smithi066 systemd[1]: Starting Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:49.815 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:49 smithi066 ceph-mon[31695]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:41:49.815 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:49 smithi066 ceph-mon[25398]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:41:49.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:49 smithi117 ceph-mon[27457]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:41:49.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:50.156 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 podman[60456]: 2024-09-10 18:41:49.927562095 +0000 UTC m=+0.196919861 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:41:50.156 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 podman[60456]: 2024-09-10 18:41:50.046675133 +0000 UTC m=+0.316032893 volume create bd5e8d442b029cab540aa8fcc1d6c5432609e3950aaf3ce714bc0035ba31cd9a 2024-09-10T18:41:50.156 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 podman[60456]: 2024-09-10 18:41:50.155446233 +0000 UTC m=+0.424803995 container create 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:50.702 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 podman[60456]: 2024-09-10 18:41:50.441322643 +0000 UTC m=+0.710680399 container init 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 podman[60456]: 2024-09-10 18:41:50.444479923 +0000 UTC m=+0.713837679 container start 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.458Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.459Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.460Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.66 port=9094 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.461Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.489Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.489Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.492Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:50.492Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:41:50.703 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 bash[60456]: 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f 2024-09-10T18:41:50.704 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:50 smithi066 systemd[1]: Started Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:50.704 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:50 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:50.815 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:50 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:41:50.978 INFO:teuthology.orchestra.run.smithi066.stdout:Backend: cephadm 2024-09-10T18:41:50.978 INFO:teuthology.orchestra.run.smithi066.stdout:Available: Yes 2024-09-10T18:41:50.978 INFO:teuthology.orchestra.run.smithi066.stdout:Paused: No 2024-09-10T18:41:51.725 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:51.725 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:51.725 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:51.725 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:41:51.725 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: Reconfiguring daemon prometheus.a on smithi117 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[31695]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: Reconfiguring daemon prometheus.a on smithi117 2024-09-10T18:41:51.726 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:51 smithi066 ceph-mon[25398]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:51.751 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T18:41:52.045 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: Reconfiguring daemon prometheus.a on smithi117 2024-09-10T18:41:52.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:51 smithi117 ceph-mon[27457]: from='client.14733 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:52.728 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:41:52 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:41:52.462Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000107993s 2024-09-10T18:41:52.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:52 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:52.729 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:52 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:52.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:52 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:52.729 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:52 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:52.753 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:52 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:52.753 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:52 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:41:53.711 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 systemd[1]: Stopping Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[31695]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[25398]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:53.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:53.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:53.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:53 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:54.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-mon[27457]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 862 B/s rd, 0 op/s 2024-09-10T18:41:54.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:54.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:54.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:41:54.066 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.818Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.819Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.819Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[49001]: ts=2024-09-10T18:41:53.819Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:41:54.067 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:53 smithi117 podman[53285]: 2024-09-10 18:41:53.828565529 +0000 UTC m=+0.127717075 container died c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:54.436 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:54 smithi117 podman[53285]: 2024-09-10 18:41:54.226047442 +0000 UTC m=+0.525198978 container cleanup c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:54.436 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:54 smithi117 bash[53285]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:agent.smithi066 smithi066 running 0s ago 6m - - 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:agent.smithi117 smithi117 running 9s ago 6m - - 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:alertmanager.a smithi066 *:9093,9094 error 0s ago 45s 23.7M - 0.25.0 c8568f914cd2 e77077d9c178 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:grafana.a smithi117 *:3000 running (32s) 9s ago 30s 77.7M - 9.4.7 954c08fa6188 d893f555d22a 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:iscsi.iscsi.a smithi117 *:5000 running (106s) 9s ago 105s 56.3M - 3.8 4c55dd409611 04044b78827c 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.x smithi117 *:8443 running (5m) 9s ago 5m 398M - 17.2.7-1606-g5be57395 4c55dd409611 4e20110491d2 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:mgr.y smithi066 *:9283,8765 running (7m) 0s ago 7m 449M - 17.2.7-1606-g5be57395 4c55dd409611 415ea3f1a802 2024-09-10T18:41:54.528 INFO:teuthology.orchestra.run.smithi066.stdout:mon.a smithi066 running (7m) 0s ago 7m 48.8M 2048M 17.2.7-1606-g5be57395 4c55dd409611 a3b0caa5ed21 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:mon.b smithi117 running (5m) 9s ago 5m 43.8M 2048M 17.2.7-1606-g5be57395 4c55dd409611 103585c9d90a 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:mon.c smithi066 running (5m) 0s ago 5m 44.7M 2048M 17.2.7-1606-g5be57395 4c55dd409611 2be562bd89a8 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:node-exporter.a smithi066 *:9100 running (22s) 0s ago 21s 3581k - 1.5.0 0da6a335fe13 d0e424adf5c8 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:node-exporter.b smithi117 *:9100 running (18s) 9s ago 18s 3934k - 1.5.0 0da6a335fe13 9f4428ee0322 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.0 smithi066 running (5m) 0s ago 5m 46.5M 4096M 17.2.7-1606-g5be57395 4c55dd409611 2ed112cbdaf2 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.1 smithi066 running (4m) 0s ago 4m 48.4M 4096M 17.2.7-1606-g5be57395 4c55dd409611 c5a64f8ae365 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.2 smithi066 running (4m) 0s ago 4m 43.9M 4096M 17.2.7-1606-g5be57395 4c55dd409611 05195728f5bd 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.3 smithi066 running (3m) 0s ago 3m 44.7M 4096M 17.2.7-1606-g5be57395 4c55dd409611 1a42dd2a2cc1 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.4 smithi117 running (3m) 9s ago 3m 45.0M 4096M 17.2.7-1606-g5be57395 4c55dd409611 43dc82ead185 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.5 smithi117 running (3m) 9s ago 3m 44.1M 4096M 17.2.7-1606-g5be57395 4c55dd409611 d2e619f866bc 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.6 smithi117 running (2m) 9s ago 2m 45.5M 4096M 17.2.7-1606-g5be57395 4c55dd409611 bdf2ec0e3df8 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:osd.7 smithi117 running (2m) 9s ago 2m 44.5M 4096M 17.2.7-1606-g5be57395 4c55dd409611 cc66b0e0fbcc 2024-09-10T18:41:54.529 INFO:teuthology.orchestra.run.smithi066.stdout:prometheus.a smithi117 *:9095 running (84s) 9s ago 83s 28.5M - 2.43.0 a07b618ecd1d c72f7a3d06c8 2024-09-10T18:41:54.530 INFO:teuthology.orchestra.run.smithi066.stdout:rgw.foo.a smithi066 *:80 running (2m) 0s ago 2m 66.8M - 17.2.7-1606-g5be57395 4c55dd409611 e5885dd8287e 2024-09-10T18:41:54.715 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:54 smithi117 podman[53297]: 2024-09-10 18:41:54.435664694 +0000 UTC m=+0.608531619 container remove c72f7a3d06c8a41dbd0667e82e8ca322d29b954e0fa06921d39c6afa9895f1c8 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:54.878 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:54 smithi066 ceph-mon[31695]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:54.878 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:54 smithi066 ceph-mon[31695]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:54.878 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:54 smithi066 ceph-mon[25398]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:54.879 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:54 smithi066 ceph-mon[25398]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:55.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:54 smithi117 ceph-mon[27457]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:55.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:54 smithi117 ceph-mon[27457]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:55.198 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T18:41:55.456 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:55.694 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:41:55.694 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 systemd[1]: Stopped Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:55.695 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.047s CPU time. 2024-09-10T18:41:55.695 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 systemd[1]: Starting Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:41:55.857 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:55 smithi066 ceph-mon[31695]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:55.857 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:55 smithi066 ceph-mon[25398]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:55.946 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-mon[27457]: from='client.24638 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:55.946 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 podman[53423]: 2024-09-10 18:41:55.693527957 +0000 UTC m=+0.102538626 container create 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:55.946 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 podman[53423]: 2024-09-10 18:41:55.60786257 +0000 UTC m=+0.016873251 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:41:55.946 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 podman[53423]: 2024-09-10 18:41:55.912583195 +0000 UTC m=+0.321593861 container init 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:55.946 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 podman[53423]: 2024-09-10 18:41:55.915250817 +0000 UTC m=+0.324261494 container start 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:41:55.946 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.939Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.939Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.939Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi117 (none))" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.939Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.939Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.941Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.941Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.942Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.942Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.277µs 2024-09-10T18:41:55.947 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:41:55.948 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STOPPING 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STOPPED 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STARTING 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Serving on http://:::9283 2024-09-10T18:41:56.170 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STARTED 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.945Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=29.606µs wal_replay_duration=660.277µs wbl_replay_duration=137ns total_replay_duration=714.974µs 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.946Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.946Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.946Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.961Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=14.419483ms db_storage=686ns remote_storage=1.08µs web_handler=304ns query_engine=537ns scrape=357.281µs scrape_sd=27.544µs notify=24.023µs notify_sd=6.197µs rules=13.748765ms tracing=5.65µs 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.961Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:41:56.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:41:55.961Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:41:56.273 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 bash[53423]: 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 2024-09-10T18:41:56.273 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:41:55 smithi117 systemd[1]: Started Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:41:56.471 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STOPPING 2024-09-10T18:41:56.722 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:41:56.722 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STOPPED 2024-09-10T18:41:56.722 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STARTING 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Serving on http://:::9283 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STARTED 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:56] ENGINE Bus STOPPING 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:56.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.980 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:56.981 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:56 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.052 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:57.052 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.052 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.052 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.053 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:41:57.054 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:41:57.054 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:41:57.054 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:56 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:agent 2/2 12s ago 80s * 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:alertmanager ?:9093,9094 0/1 3s ago 101s smithi066=a;count:1 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:grafana ?:3000 1/1 12s ago 98s smithi117=a;count:1 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:iscsi.datapool ?:5000 1/1 12s ago 113s smithi117=iscsi.a;count:1 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:mgr 2/2 12s ago 5m smithi066=y;smithi117=x;count:2 2024-09-10T18:41:57.113 INFO:teuthology.orchestra.run.smithi066.stdout:mon 3/3 12s ago 6m smithi066:172.21.15.66=a;smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c;smithi117:172.21.15.117=b;count:3 2024-09-10T18:41:57.114 INFO:teuthology.orchestra.run.smithi066.stdout:node-exporter ?:9100 2/2 12s ago 105s smithi066=a;smithi117=b;count:2 2024-09-10T18:41:57.114 INFO:teuthology.orchestra.run.smithi066.stdout:osd 8 12s ago - 2024-09-10T18:41:57.114 INFO:teuthology.orchestra.run.smithi066.stdout:prometheus ?:9095 0/1 - 109s smithi117=a;count:1 2024-09-10T18:41:57.115 INFO:teuthology.orchestra.run.smithi066.stdout:rgw.foo.a ?:80 1/1 3s ago 2m smithi066=foo.a;count:1 2024-09-10T18:41:57.386 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:41:57.386 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:57] ENGINE Bus STOPPED 2024-09-10T18:41:57.386 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:57] ENGINE Bus STARTING 2024-09-10T18:41:57.387 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:57] ENGINE Serving on http://:::9283 2024-09-10T18:41:57.723 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:41:57] ENGINE Bus STARTED 2024-09-10T18:41:57.857 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T18:41:57.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:57.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:57.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:57.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:57.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[31695]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:57.979 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:57.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:41:57.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-10T18:41:57.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:57.980 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:57 smithi066 ceph-mon[25398]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi066.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi117.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi117.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 678 B/s rd, 0 op/s 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 922 B/s rd, 0 op/s 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T18:41:58.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:57 smithi117 ceph-mon[27457]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:41:58.116 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:41:58.927 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:58 smithi066 ceph-mon[31695]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:58.928 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:58 smithi066 ceph-mon[25398]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:59.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:58 smithi117 ceph-mon[27457]: from='client.24644 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:41:59.730 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:41:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:41:59.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:41:59 smithi066 ceph-mon[31695]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:41:59.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:41:59 smithi066 ceph-mon[25398]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:00.001 INFO:teuthology.orchestra.run.smithi066.stdout:HOST ADDR LABELS STATUS 2024-09-10T18:42:00.001 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 172.21.15.66 2024-09-10T18:42:00.001 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 172.21.15.117 2024-09-10T18:42:00.001 INFO:teuthology.orchestra.run.smithi066.stdout:2 hosts in cluster 2024-09-10T18:42:00.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:41:59 smithi117 ceph-mon[27457]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:00.728 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:42:00 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:42:00.464Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002142709s 2024-09-10T18:42:00.797 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T18:42:01.057 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:01.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:00 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:01.179 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:00 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:01.179 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:00 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:01.848 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:01 smithi117 ceph-mon[27457]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:01.848 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:01 smithi117 ceph-mon[27457]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:02.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:01 smithi066 ceph-mon[31695]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:02.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:01 smithi066 ceph-mon[31695]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:02.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:01 smithi066 ceph-mon[25398]: from='client.24650 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:02.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:01 smithi066 ceph-mon[25398]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT53430091400BGN 372G No 9s ago Has a FileSystem, LVM detected 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme1n1 ssd Linux_3533e2d361595f882993 89.4G No 9s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme2n1 ssd Linux_75aed262aa944d09824a 89.4G No 9s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme3n1 ssd Linux_5553d225c4aa9749e4e2 89.4G No 9s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/nvme4n1 ssd Linux_fe93375e814f916c1593 89.4G No 9s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.943 INFO:teuthology.orchestra.run.smithi066.stdout:smithi066 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W4W2RJ 931G No 9s ago Has partitions 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V6400BGN 372G No 17s ago Has a FileSystem, LVM detected 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/nvme1n1 ssd Linux_f97121855214c7129da4 89.4G No 17s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/nvme2n1 ssd Linux_4d6ffbfc4584fd33863c 89.4G No 17s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/nvme3n1 ssd Linux_74bf8d7c2a1fd726ed33 89.4G No 17s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/nvme4n1 ssd Linux_2efa744e6b687b594f4e 89.4G No 17s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-09-10T18:42:02.944 INFO:teuthology.orchestra.run.smithi066.stdout:smithi117 /dev/sda hdd ATA_ST1000NM0033-9ZM173_Z1W5R17F 931G No 17s ago Has partitions 2024-09-10T18:42:03.701 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls --format yaml' 2024-09-10T18:42:03.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:03 smithi066 ceph-mon[31695]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-09-10T18:42:03.978 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:03 smithi066 ceph-mon[31695]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:03.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:03 smithi066 ceph-mon[25398]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-09-10T18:42:03.978 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:03 smithi066 ceph-mon[25398]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:03.987 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:04.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:03 smithi117 ceph-mon[27457]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 841 B/s rd, 0 op/s 2024-09-10T18:42:04.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:03 smithi117 ceph-mon[27457]: from='client.14757 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:04.709 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:04 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:42:04] "GET /metrics HTTP/1.1" 200 219772 "" "Prometheus/2.43.0" 2024-09-10T18:42:05.065 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:42:04] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.039 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.040 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:05 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:42:06.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:05 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: agent 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: agent 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout: host_pattern: '*' 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:36.670769Z' 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.523751Z' 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout: running: 2 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout: size: 2 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:events: 2024-09-10T18:42:06.325 INFO:teuthology.orchestra.run.smithi066.stdout:- 2024-09-10T18:40:38.436078Z service:agent [INFO] "service was created" 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: alertmanager 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: alertmanager 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: count: 1 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066=a 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:15.798483Z' 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.524176Z' 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: - 9093 2024-09-10T18:42:06.326 INFO:teuthology.orchestra.run.smithi066.stdout: - 9094 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: running: 0 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: size: 1 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:events: 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:- 2024-09-10T18:41:09.337008Z service:alertmanager [INFO] "service was created" 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: grafana 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: grafana 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: count: 1 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117=a 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:spec: 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout: anonymous_access: true 2024-09-10T18:42:06.327 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:19.089704Z' 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:42:05.223172Z' 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: - 3000 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: running: 1 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout: size: 1 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:events: 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:- 2024-09-10T18:41:24.422981Z service:grafana [INFO] "service was created" 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: iscsi 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:service_id: datapool 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: iscsi.datapool 2024-09-10T18:42:06.328 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: count: 1 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117=iscsi.a 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout:spec: 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: api_password: admin 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: api_port: 5000 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: api_user: admin 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: pool: datapool 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: trusted_ip_list: 172.21.15.117 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:04.088095Z' 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:42:05.223047Z' 2024-09-10T18:42:06.329 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: - 5000 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: running: 1 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: size: 1 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: mgr 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: mgr 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: count: 2 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066=y 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117=x 2024-09-10T18:42:06.330 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:36:20.174413Z' 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.523680Z' 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: running: 2 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: size: 2 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: mon 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: mon 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: count: 3 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066:172.21.15.66=a 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066:[v2:172.21.15.66:3301,v1:172.21.15.66:6790]=c 2024-09-10T18:42:06.331 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117:172.21.15.117=b 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:35:48.372957Z' 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.523551Z' 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: running: 3 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: size: 3 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: node-exporter 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: node-exporter 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: count: 2 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066=a 2024-09-10T18:42:06.332 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117=b 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:11.516084Z' 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.524237Z' 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: - 9100 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: running: 2 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout: size: 2 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:events: 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:- 2024-09-10T18:41:36.564817Z service:node-exporter [INFO] "service was created" 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: osd 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: osd 2024-09-10T18:42:06.333 INFO:teuthology.orchestra.run.smithi066.stdout:unmanaged: true 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:spec: 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: filter_logic: AND 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: objectstore: bluestore 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: container_image_id: 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.523870Z' 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: running: 8 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: size: 8 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: prometheus 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: prometheus 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.334 INFO:teuthology.orchestra.run.smithi066.stdout: count: 1 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi117=a 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:40:07.908932Z' 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:42:05.223109Z' 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: - 9095 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: running: 1 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout: size: 1 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout:events: 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout:- 2024-09-10T18:41:36.572784Z service:prometheus [INFO] "service was created" 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout:--- 2024-09-10T18:42:06.335 INFO:teuthology.orchestra.run.smithi066.stdout:service_type: rgw 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout:service_id: foo.a 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout:service_name: rgw.foo.a 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout:placement: 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: count: 1 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: hosts: 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: - smithi066=foo.a 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout:status: 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: created: '2024-09-10T18:39:55.161953Z' 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: last_refresh: '2024-09-10T18:41:53.524115Z' 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: ports: 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: - 80 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: running: 1 2024-09-10T18:42:06.336 INFO:teuthology.orchestra.run.smithi066.stdout: size: 1 2024-09-10T18:42:07.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:06 smithi117 ceph-mon[27457]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:06 smithi117 ceph-mon[27457]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:06 smithi117 ceph-mon[27457]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 724 B/s rd, 0 op/s 2024-09-10T18:42:07.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:06 smithi117 ceph-mon[27457]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:07.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:06 smithi117 ceph-mon[27457]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[31695]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[31695]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[31695]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 724 B/s rd, 0 op/s 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[31695]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[31695]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[25398]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.213 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[25398]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T18:42:07.214 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[25398]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 724 B/s rd, 0 op/s 2024-09-10T18:42:07.214 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[25398]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:07.214 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:06 smithi066 ceph-mon[25398]: from='mgr.24476 172.21.15.66:0/3478031463' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:42:07.240 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f248e234-6fa2-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-09-10T18:42:07.749 INFO:teuthology.orchestra.run.smithi066.stderr:Inferring config /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:42:08.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:07 smithi117 ceph-mon[27457]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T18:42:08.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:07 smithi066 ceph-mon[25398]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T18:42:08.228 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:07 smithi066 ceph-mon[31695]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-09-10T18:42:08.776 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:08 smithi066 ceph-mon[25398]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:08.776 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:08 smithi066 ceph-mon[31695]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:09.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:08 smithi117 ceph-mon[27457]: pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:09.815 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:42:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug there is no tcmu-runner data available 2024-09-10T18:42:10.896 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:10 smithi066 ceph-mon[31695]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:10.896 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:10 smithi066 ceph-mon[31695]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:10.897 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:10 smithi066 ceph-mon[25398]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:10.897 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:10 smithi066 ceph-mon[25398]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:11.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:10 smithi117 ceph-mon[27457]: pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T18:42:11.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:10 smithi117 ceph-mon[27457]: from='client.24466 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:42:11.316 INFO:teuthology.orchestra.run.smithi066.stdout:osd 8 17s ago - 2024-09-10T18:42:12.093 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T18:42:12.102 INFO:tasks.cephadm:Teardown begin 2024-09-10T18:42:12.102 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:42:12.136 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:42:12.171 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T18:42:12.172 DEBUG:teuthology.orchestra.run.smithi066:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:42:12.195 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:42:12.230 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T18:42:12.231 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T18:42:12.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:42:12.356 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:12 smithi066 systemd[1]: Stopping Ceph mon.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:13.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:12 smithi117 ceph-mon[27457]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-09-10T18:42:13.066 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:12 smithi117 ceph-mon[27457]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:13.176 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-mon[31695]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-mon[31695]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:12] ENGINE Bus STOPPING 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-mon[25398]: pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 835 B/s rd, 0 op/s 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-mon[25398]: from='client.24668 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a[25394]: 2024-09-10T18:42:12.950+0000 7fa0423fa640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:13.177 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:12 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a[25394]: 2024-09-10T18:42:12.950+0000 7fa0423fa640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T18:42:13.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:13 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:42:13.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:13 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:13] ENGINE Bus STOPPED 2024-09-10T18:42:13.478 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:13 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:13] ENGINE Bus STARTING 2024-09-10T18:42:13.865 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:13 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:13] ENGINE Serving on http://:::9283 2024-09-10T18:42:13.865 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:13 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:13] ENGINE Bus STARTED 2024-09-10T18:42:13.866 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:13 smithi066 podman[62348]: 2024-09-10 18:42:13.612179414 +0000 UTC m=+0.773043055 container died a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T18:42:14.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:14 smithi066 podman[62348]: 2024-09-10 18:42:14.06399007 +0000 UTC m=+1.224853753 container cleanup a3b0caa5ed211b2987dc30980ee3fedfa15c4f05b7d38b4a8a6c9ed2614b68d9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:14.228 INFO:journalctl@ceph.mon.a.smithi066.stdout:Sep 10 18:42:14 smithi066 bash[62348]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:42:14.495 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T18:42:14.541 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:42:14] "GET /metrics HTTP/1.1" 200 219769 "" "Prometheus/2.43.0" 2024-09-10T18:42:14.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:14.543 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T18:42:14.543 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T18:42:14.543 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.c 2024-09-10T18:42:14.829 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:14 smithi066 systemd[1]: Stopping Ceph mon.c for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:15.023 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x[29582]: ::ffff:172.21.15.117 - - [10/Sep/2024:18:42:14] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:14] ENGINE Bus STOPPING 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:14] ENGINE Bus STOPPED 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:14] ENGINE Bus STARTING 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:15 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:15] ENGINE Serving on http://:::9283 2024-09-10T18:42:15.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:15 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: [10/Sep/2024:18:42:15] ENGINE Bus STARTED 2024-09-10T18:42:15.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c[31689]: 2024-09-10T18:42:14.826+0000 7f8de6b20640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:15.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:14 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c[31689]: 2024-09-10T18:42:14.826+0000 7f8de6b20640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:42:15.229 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:14 smithi066 podman[62551]: 2024-09-10 18:42:14.960227285 +0000 UTC m=+0.228184576 container died 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T18:42:15.697 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:15 smithi066 podman[62551]: 2024-09-10 18:42:15.461482551 +0000 UTC m=+0.729439837 container cleanup 2be562bd89a87515e23fdc40d089741d7912b53764123a57840fa9d355ebf8aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:42:15.697 INFO:journalctl@ceph.mon.c.smithi066.stdout:Sep 10 18:42:15 smithi066 bash[62551]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-c 2024-09-10T18:42:15.853 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T18:42:15.890 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:15.890 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T18:42:15.890 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T18:42:15.890 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.b 2024-09-10T18:42:16.065 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:15 smithi117 systemd[1]: Stopping Ceph mon.b for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:16.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b[27444]: 2024-09-10T18:42:16.424+0000 7efdbf8dd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:16.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b[27444]: 2024-09-10T18:42:16.424+0000 7efdbf8dd640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T18:42:16.816 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:16 smithi117 podman[54381]: 2024-09-10 18:42:16.485977286 +0000 UTC m=+0.162449574 container died 103585c9d90a40a1be44d7c9105351c179ec558958a324f7455855e38a7d5ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T18:42:17.172 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:16 smithi117 podman[54381]: 2024-09-10 18:42:16.841277253 +0000 UTC m=+0.517749540 container cleanup 103585c9d90a40a1be44d7c9105351c179ec558958a324f7455855e38a7d5ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:17.172 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:16 smithi117 bash[54381]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b 2024-09-10T18:42:17.884 INFO:journalctl@ceph.mon.b.smithi117.stdout:Sep 10 18:42:17 smithi117 podman[54410]: 2024-09-10 18:42:17.629982934 +0000 UTC m=+1.143338566 container remove 103585c9d90a40a1be44d7c9105351c179ec558958a324f7455855e38a7d5ae9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T18:42:17.889 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T18:42:17.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:17.942 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T18:42:17.942 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T18:42:17.942 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:42:18.228 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:17 smithi066 systemd[1]: Stopping Ceph mgr.y for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:18.594 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:18 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:42:18.264+0000 7f39ed6ae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:18.594 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:18 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y[25627]: 2024-09-10T18:42:18.264+0000 7f39ed6ae640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T18:42:18.595 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:18 smithi066 podman[62683]: 2024-09-10 18:42:18.297040356 +0000 UTC m=+0.149914332 container died 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:42:18.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:18 smithi066 podman[62683]: 2024-09-10 18:42:18.758515561 +0000 UTC m=+0.611389529 container cleanup 415ea3f1a8028424abf2b001021b9bfff5b16d44a6caceaa98eb6f083d898d88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:18.978 INFO:journalctl@ceph.mgr.y.smithi066.stdout:Sep 10 18:42:18 smithi066 bash[62683]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-y 2024-09-10T18:42:19.196 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T18:42:19.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:19.234 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T18:42:19.234 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T18:42:19.234 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x 2024-09-10T18:42:19.565 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:19 smithi117 systemd[1]: Stopping Ceph mgr.x for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:19.894 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:19 smithi117 podman[54579]: 2024-09-10 18:42:19.570124669 +0000 UTC m=+0.144164065 container died 4e20110491d2dd674ad6bf1c16fad6548b9e5b33e8774d20b8ccbe65ccbeb024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T18:42:20.206 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:19 smithi117 podman[54579]: 2024-09-10 18:42:19.999387575 +0000 UTC m=+0.573426961 container cleanup 4e20110491d2dd674ad6bf1c16fad6548b9e5b33e8774d20b8ccbe65ccbeb024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T18:42:20.206 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:19 smithi117 bash[54579]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x 2024-09-10T18:42:20.444 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T18:42:20.466 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:20 smithi117 podman[54590]: 2024-09-10 18:42:20.201152923 +0000 UTC m=+0.625494547 container remove 4e20110491d2dd674ad6bf1c16fad6548b9e5b33e8774d20b8ccbe65ccbeb024 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:42:20.467 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:20 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:42:20.467 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:20 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x.service: Failed with result 'exit-code'. 2024-09-10T18:42:20.467 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:20 smithi117 systemd[1]: Stopped Ceph mgr.x for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:42:20.467 INFO:journalctl@ceph.mgr.x.smithi117.stdout:Sep 10 18:42:20 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@mgr.x.service: Consumed 10.252s CPU time. 2024-09-10T18:42:20.481 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:20.482 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T18:42:20.482 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T18:42:20.482 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.0 2024-09-10T18:42:20.781 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:20 smithi066 systemd[1]: Stopping Ceph osd.0 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:21.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0[36230]: 2024-09-10T18:42:20.778+0000 7f38365a6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:21.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0[36230]: 2024-09-10T18:42:20.778+0000 7f38365a6640 -1 osd.0 71 *** Got signal Terminated *** 2024-09-10T18:42:21.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:20 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0[36230]: 2024-09-10T18:42:20.778+0000 7f38365a6640 -1 osd.0 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:42:26.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:26 smithi066 podman[62829]: 2024-09-10 18:42:26.15166559 +0000 UTC m=+5.473249658 container died 2ed112cbdaf2718172043584e924eff7715152b732fd0ba8d00add81d48bd459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T18:42:26.728 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:26 smithi066 podman[62829]: 2024-09-10 18:42:26.616023661 +0000 UTC m=+5.937607722 container cleanup 2ed112cbdaf2718172043584e924eff7715152b732fd0ba8d00add81d48bd459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T18:42:26.728 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:26 smithi066 bash[62829]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0 2024-09-10T18:42:27.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:26 smithi066 podman[62995]: 2024-09-10 18:42:26.834475494 +0000 UTC m=+0.710585967 container remove 2ed112cbdaf2718172043584e924eff7715152b732fd0ba8d00add81d48bd459 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True) 2024-09-10T18:42:27.949 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:27 smithi066 podman[63087]: 2024-09-10 18:42:27.583737125 +0000 UTC m=+0.016496802 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:42:27.950 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:27 smithi066 podman[63087]: 2024-09-10 18:42:27.721709996 +0000 UTC m=+0.154469656 container create fa7350f1d8d5028b22b1c0ad6bbf00bf55d6413136f7d87e9bd17416a55f1408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T18:42:28.228 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:27 smithi066 podman[63087]: 2024-09-10 18:42:27.948882517 +0000 UTC m=+0.381642187 container init fa7350f1d8d5028b22b1c0ad6bbf00bf55d6413136f7d87e9bd17416a55f1408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-10T18:42:28.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:27 smithi066 podman[63087]: 2024-09-10 18:42:27.952398105 +0000 UTC m=+0.385157772 container start fa7350f1d8d5028b22b1c0ad6bbf00bf55d6413136f7d87e9bd17416a55f1408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T18:42:28.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:28 smithi066 podman[63087]: 2024-09-10 18:42:28.04087741 +0000 UTC m=+0.473637088 container attach fa7350f1d8d5028b22b1c0ad6bbf00bf55d6413136f7d87e9bd17416a55f1408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T18:42:28.229 INFO:journalctl@ceph.osd.0.smithi066.stdout:Sep 10 18:42:28 smithi066 podman[63087]: 2024-09-10 18:42:28.142205585 +0000 UTC m=+0.574965246 container died fa7350f1d8d5028b22b1c0ad6bbf00bf55d6413136f7d87e9bd17416a55f1408 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:28.931 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T18:42:28.976 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:28.976 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T18:42:28.976 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T18:42:28.976 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.1 2024-09-10T18:42:29.358 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:29 smithi066 systemd[1]: Stopping Ceph osd.1 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:29.693 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:29 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1[39621]: 2024-09-10T18:42:29.355+0000 7f7cbb4cc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:29.694 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:29 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1[39621]: 2024-09-10T18:42:29.355+0000 7f7cbb4cc640 -1 osd.1 71 *** Got signal Terminated *** 2024-09-10T18:42:29.694 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:29 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1[39621]: 2024-09-10T18:42:29.355+0000 7f7cbb4cc640 -1 osd.1 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:42:34.978 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:34 smithi066 podman[63191]: 2024-09-10 18:42:34.723626357 +0000 UTC m=+5.551589959 container died c5a64f8ae365f763087eea8e151cd13dd0c003e8e2e83324609fdd54ebe8df0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T18:42:35.478 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63191]: 2024-09-10 18:42:35.13214818 +0000 UTC m=+5.960111824 container cleanup c5a64f8ae365f763087eea8e151cd13dd0c003e8e2e83324609fdd54ebe8df0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2) 2024-09-10T18:42:35.478 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 bash[63191]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1 2024-09-10T18:42:35.478 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63765]: 2024-09-10 18:42:35.350035062 +0000 UTC m=+0.622714693 container remove c5a64f8ae365f763087eea8e151cd13dd0c003e8e2e83324609fdd54ebe8df0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T18:42:35.937 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63856]: 2024-09-10 18:42:35.568479477 +0000 UTC m=+0.016323530 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:42:35.937 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63856]: 2024-09-10 18:42:35.675777109 +0000 UTC m=+0.123621191 container create 88eec45f0a111e8b9ccca41bdcb1a3d1b0defb371c6919fe01f5371c6b6b187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:35.937 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63856]: 2024-09-10 18:42:35.869567803 +0000 UTC m=+0.317411843 container init 88eec45f0a111e8b9ccca41bdcb1a3d1b0defb371c6919fe01f5371c6b6b187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS) 2024-09-10T18:42:35.937 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63856]: 2024-09-10 18:42:35.87315988 +0000 UTC m=+0.321003927 container start 88eec45f0a111e8b9ccca41bdcb1a3d1b0defb371c6919fe01f5371c6b6b187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T18:42:35.937 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:35 smithi066 podman[63856]: 2024-09-10 18:42:35.936062194 +0000 UTC m=+0.383906248 container attach 88eec45f0a111e8b9ccca41bdcb1a3d1b0defb371c6919fe01f5371c6b6b187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:42:36.228 INFO:journalctl@ceph.osd.1.smithi066.stdout:Sep 10 18:42:36 smithi066 podman[63856]: 2024-09-10 18:42:36.057569828 +0000 UTC m=+0.505413879 container died 88eec45f0a111e8b9ccca41bdcb1a3d1b0defb371c6919fe01f5371c6b6b187f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-10T18:42:36.793 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T18:42:36.837 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:36.837 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T18:42:36.837 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T18:42:36.837 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.2 2024-09-10T18:42:37.186 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:36 smithi066 systemd[1]: Stopping Ceph osd.2 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:37.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2[43257]: 2024-09-10T18:42:37.184+0000 7f371b3f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:37.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2[43257]: 2024-09-10T18:42:37.184+0000 7f371b3f2640 -1 osd.2 71 *** Got signal Terminated *** 2024-09-10T18:42:37.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:37 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2[43257]: 2024-09-10T18:42:37.184+0000 7f371b3f2640 -1 osd.2 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:42:43.382 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:43 smithi066 podman[63960]: 2024-09-10 18:42:43.12896346 +0000 UTC m=+6.051822069 container died 05195728f5bdbc6807cbf715f5a7ceb17f350cf75d1f726e322c9345508f6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T18:42:43.728 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:43 smithi066 podman[63960]: 2024-09-10 18:42:43.571179143 +0000 UTC m=+6.494037750 container cleanup 05195728f5bdbc6807cbf715f5a7ceb17f350cf75d1f726e322c9345508f6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T18:42:43.728 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:43 smithi066 bash[63960]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2 2024-09-10T18:42:44.198 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:43 smithi066 podman[64114]: 2024-09-10 18:42:43.830844207 +0000 UTC m=+1.121559026 container remove 05195728f5bdbc6807cbf715f5a7ceb17f350cf75d1f726e322c9345508f6dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T18:42:44.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.10030598 +0000 UTC m=+0.016249938 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:42:44.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.198250092 +0000 UTC m=+0.114194049 container create 8aed0996cc21f2159a6f773d7ecb1c2adcb36288c036dc78c81eec7ce6ccbe5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T18:42:44.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.408904932 +0000 UTC m=+0.324848890 container init 8aed0996cc21f2159a6f773d7ecb1c2adcb36288c036dc78c81eec7ce6ccbe5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T18:42:44.478 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.412284991 +0000 UTC m=+0.328228957 container start 8aed0996cc21f2159a6f773d7ecb1c2adcb36288c036dc78c81eec7ce6ccbe5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T18:42:44.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.483842911 +0000 UTC m=+0.399786884 container attach 8aed0996cc21f2159a6f773d7ecb1c2adcb36288c036dc78c81eec7ce6ccbe5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:42:44.978 INFO:journalctl@ceph.osd.2.smithi066.stdout:Sep 10 18:42:44 smithi066 podman[64206]: 2024-09-10 18:42:44.597539667 +0000 UTC m=+0.513483633 container died 8aed0996cc21f2159a6f773d7ecb1c2adcb36288c036dc78c81eec7ce6ccbe5c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-10T18:42:45.165 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T18:42:45.206 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:45.207 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T18:42:45.207 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T18:42:45.207 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.3 2024-09-10T18:42:45.478 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:45 smithi066 systemd[1]: Stopping Ceph osd.3 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:45.978 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:45 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:45.568+0000 7fb948ca4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:45.978 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:45 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:45.568+0000 7fb948ca4640 -1 osd.3 71 *** Got signal Terminated *** 2024-09-10T18:42:45.978 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:45 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:45.568+0000 7fb948ca4640 -1 osd.3 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:42:47.067 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:46 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:46.631+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:48.065 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:47 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:47.830+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:48.066 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:47 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:47.634+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:48.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:47 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:47.840+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:48.728 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:48 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:48.336+0000 7fb9422b6640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.778905+0000 front 2024-09-10T18:42:25.778993+0000 (oldest deadline 2024-09-10T18:42:47.479022+0000) 2024-09-10T18:42:48.859 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:48 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:48.815+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:48.859 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:48 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:48.604+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:49.315 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:48 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:48.857+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:49.728 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:49 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:49.320+0000 7fb9422b6640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.778905+0000 front 2024-09-10T18:42:25.778993+0000 (oldest deadline 2024-09-10T18:42:47.479022+0000) 2024-09-10T18:42:49.847 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:49.588+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:49.847 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:49.809+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:50.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:49.845+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:50.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:49 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:49.846+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:50.528 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:50 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3[47049]: 2024-09-10T18:42:50.273+0000 7fb9422b6640 -1 osd.3 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.778905+0000 front 2024-09-10T18:42:25.778993+0000 (oldest deadline 2024-09-10T18:42:47.479022+0000) 2024-09-10T18:42:50.815 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:50 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:50.797+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:50.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:50 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:50.559+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:50.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:50 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:50.808+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:51.224 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:50 smithi066 podman[64309]: 2024-09-10 18:42:50.963215106 +0000 UTC m=+5.521709449 container died 1a42dd2a2cc1f0d415f06a3ae4df59ff77a47790c8660ab9d4f27eae467231c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T18:42:51.315 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:50 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:50.850+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:51.478 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:51 smithi066 podman[64309]: 2024-09-10 18:42:51.414473582 +0000 UTC m=+5.972967917 container cleanup 1a42dd2a2cc1f0d415f06a3ae4df59ff77a47790c8660ab9d4f27eae467231c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:51.478 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:51 smithi066 bash[64309]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3 2024-09-10T18:42:51.876 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:51 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:51.807+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:51.876 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:51 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:51.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:51.877 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:51 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:51.570+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:51.978 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:51 smithi066 podman[64631]: 2024-09-10 18:42:51.682467156 +0000 UTC m=+0.721557866 container remove 1a42dd2a2cc1f0d415f06a3ae4df59ff77a47790c8660ab9d4f27eae467231c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:42:52.315 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:51 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:51.874+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:52.336 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:52.066695449 +0000 UTC m=+0.103272803 container create b3b51198134e9e1c46ff294d094f76564869bf897f359c9c1c4eab4ba0523d74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True) 2024-09-10T18:42:52.336 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:51.982767959 +0000 UTC m=+0.019345251 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:42:52.336 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:52.260544076 +0000 UTC m=+0.297121377 container init b3b51198134e9e1c46ff294d094f76564869bf897f359c9c1c4eab4ba0523d74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T18:42:52.336 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:52.264083423 +0000 UTC m=+0.300660715 container start b3b51198134e9e1c46ff294d094f76564869bf897f359c9c1c4eab4ba0523d74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T18:42:52.336 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:52.335210049 +0000 UTC m=+0.371787345 container attach b3b51198134e9e1c46ff294d094f76564869bf897f359c9c1c4eab4ba0523d74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T18:42:52.728 INFO:journalctl@ceph.osd.3.smithi066.stdout:Sep 10 18:42:52 smithi066 podman[64768]: 2024-09-10 18:42:52.462127518 +0000 UTC m=+0.498704812 container died b3b51198134e9e1c46ff294d094f76564869bf897f359c9c1c4eab4ba0523d74 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:42:52.815 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:52 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:52.540+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:53.034 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T18:42:53.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:42:53.078 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T18:42:53.078 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T18:42:53.079 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.4 2024-09-10T18:42:53.109 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:52 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:52.836+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:53.109 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:52 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:52.830+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:53.110 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:52 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:52.835+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:53.375 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:53 smithi117 systemd[1]: Stopping Ceph osd.4 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:42:53.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:53.507+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:53.816 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:53.373+0000 7ff21571f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:42:53.816 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:53.373+0000 7ff21571f640 -1 osd.4 71 *** Got signal Terminated *** 2024-09-10T18:42:53.816 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:53.373+0000 7ff21571f640 -1 osd.4 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:42:54.316 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:53.872+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:54.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:53.835+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:54.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:53 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:53.877+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:54.815 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:54 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:54.505+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:55.235 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:54 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:54.881+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:55.236 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:54 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:54.852+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:55.236 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:54 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:54.846+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:55.565 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:55.475+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:55.890 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:55.804+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:55.891 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:55.820+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:56.259 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:55 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:55.889+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:56.565 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:56 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:56.458+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:56.829 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:56 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:56.791+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:57.115 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:56 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:56.934+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:57.115 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:56 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:56.826+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:57.774 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:57 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:57.444+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:58.065 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:57 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4[33671]: 2024-09-10T18:42:57.985+0000 7ff212d39640 -1 osd.4 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:24.226674+0000 front 2024-09-10T18:42:24.226816+0000 (oldest deadline 2024-09-10T18:42:47.626926+0000) 2024-09-10T18:42:58.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:57 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:57.836+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:58.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:57 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:57.773+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:58.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:57 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:57.773+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:42:58.736 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:58.404+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:58.736 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:58.404+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:42:59.065 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:58 smithi117 podman[55556]: 2024-09-10 18:42:58.88750257 +0000 UTC m=+5.612746261 container died 43dc82ead185328399fff6ff24482a3c8231aa67bf9ac50aa0137f9fc7ce712b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-10T18:42:59.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:58.731+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:42:59.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:58.731+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:42:59.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:58.809+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:42:59.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:58 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:58.809+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:42:59.709 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:59 smithi117 podman[55556]: 2024-09-10 18:42:59.44872534 +0000 UTC m=+6.173969028 container cleanup 43dc82ead185328399fff6ff24482a3c8231aa67bf9ac50aa0137f9fc7ce712b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=) 2024-09-10T18:42:59.709 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:59 smithi117 bash[55556]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4 2024-09-10T18:42:59.710 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:59.433+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:42:59.710 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:42:59.433+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:00.012 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:42:59 smithi117 podman[56008]: 2024-09-10 18:42:59.70826465 +0000 UTC m=+0.824175183 container remove 43dc82ead185328399fff6ff24482a3c8231aa67bf9ac50aa0137f9fc7ce712b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T18:43:00.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:59.753+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:00.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:42:59.753+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:00.013 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:59.815+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:00.013 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:42:59 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:42:59.815+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:00.689 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:00.424+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:00.689 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:00.424+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:01.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:00.791+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:01.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:00.791+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:01.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:00.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:01.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:00 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:00.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:01.566 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:01 smithi117 podman[56311]: 2024-09-10 18:43:01.194421211 +0000 UTC m=+0.017071481 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:43:01.566 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:01 smithi117 podman[56311]: 2024-09-10 18:43:01.299395182 +0000 UTC m=+0.122045438 container create 146dd654f4e761fcf46d8cc6faa0c7900b3e5967998bb1415bd445792bf5b3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:43:01.566 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:01.470+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:01.566 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:01.470+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:02.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:01.755+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:02.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:01.755+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:02.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:01.817+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:02.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:01 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:01.817+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:02.513 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:02 smithi117 podman[56311]: 2024-09-10 18:43:02.147130856 +0000 UTC m=+0.969781112 container init 146dd654f4e761fcf46d8cc6faa0c7900b3e5967998bb1415bd445792bf5b3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux ) 2024-09-10T18:43:02.513 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:02 smithi117 podman[56311]: 2024-09-10 18:43:02.150675786 +0000 UTC m=+0.973326043 container start 146dd654f4e761fcf46d8cc6faa0c7900b3e5967998bb1415bd445792bf5b3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T18:43:02.513 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:02 smithi117 podman[56311]: 2024-09-10 18:43:02.221922249 +0000 UTC m=+1.044572512 container attach 146dd654f4e761fcf46d8cc6faa0c7900b3e5967998bb1415bd445792bf5b3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:43:02.513 INFO:journalctl@ceph.osd.4.smithi117.stdout:Sep 10 18:43:02 smithi117 podman[56311]: 2024-09-10 18:43:02.329981151 +0000 UTC m=+1.152631418 container died 146dd654f4e761fcf46d8cc6faa0c7900b3e5967998bb1415bd445792bf5b3fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-10T18:43:02.811 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:02.712+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:02.812 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:02.712+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:02.812 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:02.511+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:02.812 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:02.511+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:03.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:02.810+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:03.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:02 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:02.810+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:03.186 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T18:43:03.269 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:03.269 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T18:43:03.269 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T18:43:03.269 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.5 2024-09-10T18:43:03.559 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 systemd[1]: Stopping Ceph osd.5 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:03.815 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:03.803+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:03.803+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:03.803+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:03.719+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:03.719+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:03.719+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:03.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:03.557+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:03.817 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:03.557+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:03.817 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:03.807+0000 7f3256fae640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:43:03.817 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:03.807+0000 7f3256fae640 -1 osd.5 71 *** Got signal Terminated *** 2024-09-10T18:43:03.817 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:03 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:03.807+0000 7f3256fae640 -1 osd.5 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:43:04.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:04.557+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:04.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:04.557+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:04.816 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:04.756+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:04.816 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:04.756+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:04.816 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:04.756+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:04.817 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:04.711+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:04.817 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:04.711+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:04.817 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:04 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:04.711+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:05.797 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:05.511+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:05.797 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:05.511+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:05.797 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:05.757+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:05.797 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:05.757+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:05.797 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:05.757+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:06.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:05.795+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:06.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:05.795+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:06.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:05 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:05.795+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:06.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:06.749+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:06.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:06.749+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:06.816 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:06.749+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:06.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:06.547+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:06.816 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:06.547+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:06.817 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:06.547+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:40.961074+0000 front 2024-09-10T18:42:40.960973+0000 (oldest deadline 2024-09-10T18:43:05.660788+0000) 2024-09-10T18:43:06.817 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:06.753+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:06.817 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:06.753+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:06.817 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:06 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:06.753+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:07.752 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:07.498+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:07.752 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:07.498+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:07.752 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:07.498+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:40.961074+0000 front 2024-09-10T18:42:40.960973+0000 (oldest deadline 2024-09-10T18:43:05.660788+0000) 2024-09-10T18:43:08.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:07.750+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:08.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:07.750+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:08.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:07.750+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:08.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:07.769+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:08.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:07.769+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:08.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:07 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:07.769+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:08.790 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:08.508+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:20.157398+0000 front 2024-09-10T18:42:20.157467+0000 (oldest deadline 2024-09-10T18:42:46.557609+0000) 2024-09-10T18:43:08.790 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:08.508+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:34.059776+0000 front 2024-09-10T18:42:34.059677+0000 (oldest deadline 2024-09-10T18:42:57.559699+0000) 2024-09-10T18:43:08.791 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5[37292]: 2024-09-10T18:43:08.508+0000 7f32535c6640 -1 osd.5 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:40.961074+0000 front 2024-09-10T18:42:40.960973+0000 (oldest deadline 2024-09-10T18:43:05.660788+0000) 2024-09-10T18:43:09.065 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:08.789+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:09.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:08.789+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:09.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:08.789+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:09.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:08.800+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:09.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:08.801+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:09.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:08 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:08.801+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:09.434 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:09 smithi117 podman[56545]: 2024-09-10 18:43:09.157241629 +0000 UTC m=+5.473509266 container died d2e619f866bc5d628d61bc827fbef5bfe1d4e6c679b2d0c1e1a41769ed926d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:43:09.798 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:09 smithi117 podman[56545]: 2024-09-10 18:43:09.615144932 +0000 UTC m=+5.931412565 container cleanup d2e619f866bc5d628d61bc827fbef5bfe1d4e6c679b2d0c1e1a41769ed926d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:43:09.799 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:09 smithi117 bash[56545]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5 2024-09-10T18:43:10.066 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:09 smithi117 podman[56573]: 2024-09-10 18:43:09.916974826 +0000 UTC m=+0.755303007 container remove d2e619f866bc5d628d61bc827fbef5bfe1d4e6c679b2d0c1e1a41769ed926d62 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T18:43:10.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:09.796+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:10.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:09.796+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:10.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:09.796+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:10.067 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:09.827+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:10.067 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:09.827+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:10.067 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:09 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:09.827+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:10.496 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.126028278 +0000 UTC m=+0.015399043 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:43:10.496 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.242879071 +0000 UTC m=+0.132249900 container create ed9d0fbeca7d8916f6e601f8a87960bcd08bea8710b06b6702c382d5494d086f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-10T18:43:10.497 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.49559681 +0000 UTC m=+0.384967574 container init ed9d0fbeca7d8916f6e601f8a87960bcd08bea8710b06b6702c382d5494d086f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:43:10.783 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.499225593 +0000 UTC m=+0.388596357 container start ed9d0fbeca7d8916f6e601f8a87960bcd08bea8710b06b6702c382d5494d086f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:43:10.783 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.578657697 +0000 UTC m=+0.468028476 container attach ed9d0fbeca7d8916f6e601f8a87960bcd08bea8710b06b6702c382d5494d086f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T18:43:10.783 INFO:journalctl@ceph.osd.5.smithi117.stdout:Sep 10 18:43:10 smithi117 podman[56659]: 2024-09-10 18:43:10.678889347 +0000 UTC m=+0.568260118 container died ed9d0fbeca7d8916f6e601f8a87960bcd08bea8710b06b6702c382d5494d086f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T18:43:11.033 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:10.781+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:11.033 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:10.781+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:11.033 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:10.781+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:11.033 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:10.866+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:11.034 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:10.866+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:11.034 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:10 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:10.866+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:11.397 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T18:43:11.441 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:11.441 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T18:43:11.441 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T18:43:11.441 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.6 2024-09-10T18:43:11.771 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 systemd[1]: Stopping Ceph osd.6 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:12.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:11.915+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:11.915+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:11.915+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.769+0000 7fcde2898640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.769+0000 7fcde2898640 -1 osd.6 71 *** Got signal Terminated *** 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.769+0000 7fcde2898640 -1 osd.6 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.777+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.777+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.777+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:12.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:11 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:11.777+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:46.960999+0000 front 2024-09-10T18:42:46.960972+0000 (oldest deadline 2024-09-10T18:43:11.060900+0000) 2024-09-10T18:43:13.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:12.901+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:12.901+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:12.901+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:12.805+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:12.805+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:12.805+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:13.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:12 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:12.805+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:46.960999+0000 front 2024-09-10T18:42:46.960972+0000 (oldest deadline 2024-09-10T18:43:11.060900+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:13.806+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:13.806+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:13.806+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:13.806+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:46.960999+0000 front 2024-09-10T18:42:46.960972+0000 (oldest deadline 2024-09-10T18:43:11.060900+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:13.856+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:13.856+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:14.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:13 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:13.856+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:15.315 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:14.818+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:14.818+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:14.818+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:14.839+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:14.839+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:14.839+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:15.316 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:14 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:14.839+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:46.960999+0000 front 2024-09-10T18:42:46.960972+0000 (oldest deadline 2024-09-10T18:43:11.060900+0000) 2024-09-10T18:43:16.043 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:15.889+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:21.857558+0000 front 2024-09-10T18:42:21.857644+0000 (oldest deadline 2024-09-10T18:42:47.157572+0000) 2024-09-10T18:43:16.043 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:15.889+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:33.559183+0000 front 2024-09-10T18:42:33.559220+0000 (oldest deadline 2024-09-10T18:42:58.258955+0000) 2024-09-10T18:43:16.043 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:15.889+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:38.259525+0000 front 2024-09-10T18:42:38.259635+0000 (oldest deadline 2024-09-10T18:43:02.859865+0000) 2024-09-10T18:43:16.043 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6[40646]: 2024-09-10T18:43:15.889+0000 7fcddeeb0640 -1 osd.6 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:46.960999+0000 front 2024-09-10T18:42:46.960972+0000 (oldest deadline 2024-09-10T18:43:11.060900+0000) 2024-09-10T18:43:16.043 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:15.785+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:16.044 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:15.785+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:16.044 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:15.785+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:16.044 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:15 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:15.785+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:17.058 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:16.759+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:17.058 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:16.759+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:17.058 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:16.759+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:17.058 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:16 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:16.759+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:17.630 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:17 smithi117 podman[56762]: 2024-09-10 18:43:17.338282077 +0000 UTC m=+5.664539566 container died bdf2ec0e3df836d04b3d813cfbde44a3a2f8248d40c3ec3dbe923d02bec58b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T18:43:17.896 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:17 smithi117 podman[56762]: 2024-09-10 18:43:17.811579837 +0000 UTC m=+6.137837371 container cleanup bdf2ec0e3df836d04b3d813cfbde44a3a2f8248d40c3ec3dbe923d02bec58b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T18:43:17.896 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:17 smithi117 bash[56762]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6 2024-09-10T18:43:17.896 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:17 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:17.791+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:17.897 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:17 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:17.791+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:17.897 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:17 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:17.791+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:17.897 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:17 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:17.791+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:18.510 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:18 smithi117 podman[56835]: 2024-09-10 18:43:18.172037925 +0000 UTC m=+1.014587322 container remove bdf2ec0e3df836d04b3d813cfbde44a3a2f8248d40c3ec3dbe923d02bec58b8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:43:18.807 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:18 smithi117 podman[56977]: 2024-09-10 18:43:18.41229489 +0000 UTC m=+0.015907718 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:43:18.807 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:18 smithi117 podman[56977]: 2024-09-10 18:43:18.556702732 +0000 UTC m=+0.160315551 container create 8dcef6a8fe36b392c8af5d8bbb3844a183a3f1962a37a86f08c0cc7dfd967f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:43:19.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:18 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:18.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:19.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:18 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:18.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:19.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:18 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:18.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:19.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:18 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:18.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:19.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:19 smithi117 podman[56977]: 2024-09-10 18:43:19.346346405 +0000 UTC m=+0.949959225 container init 8dcef6a8fe36b392c8af5d8bbb3844a183a3f1962a37a86f08c0cc7dfd967f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T18:43:19.526 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:19 smithi117 podman[56977]: 2024-09-10 18:43:19.34963286 +0000 UTC m=+0.953245679 container start 8dcef6a8fe36b392c8af5d8bbb3844a183a3f1962a37a86f08c0cc7dfd967f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:43:19.527 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:19 smithi117 podman[56977]: 2024-09-10 18:43:19.42108461 +0000 UTC m=+1.024697451 container attach 8dcef6a8fe36b392c8af5d8bbb3844a183a3f1962a37a86f08c0cc7dfd967f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T18:43:19.797 INFO:journalctl@ceph.osd.6.smithi117.stdout:Sep 10 18:43:19 smithi117 podman[56977]: 2024-09-10 18:43:19.5259484 +0000 UTC m=+1.129561246 container died 8dcef6a8fe36b392c8af5d8bbb3844a183a3f1962a37a86f08c0cc7dfd967f70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T18:43:20.050 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:19.796+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:20.051 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:19.796+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:20.051 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:19.796+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:20.051 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:19.796+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:20.051 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:19 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:19.796+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:20.070 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T18:43:20.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:20.113 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T18:43:20.113 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T18:43:20.113 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.7 2024-09-10T18:43:20.315 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 systemd[1]: Stopping Ceph osd.7 for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:20.750 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.418+0000 7f27296ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:43:20.750 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.418+0000 7f27296ea640 -1 osd.7 71 *** Got signal Terminated *** 2024-09-10T18:43:20.750 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.418+0000 7f27296ea640 -1 osd.7 71 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T18:43:21.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.758+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:21.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.758+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:21.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.758+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:21.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.758+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:21.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:20 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:20.758+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:22.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:21 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:21.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:22.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:21 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:21.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:22.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:21 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:21.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:22.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:21 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:21.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:22.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:21 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:21.781+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:23.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:22.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:23.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:22.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:23.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:22.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:23.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:22.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:23.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:22 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:22.810+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:24.011 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:23.829+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:24.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:23.829+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:24.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:23.829+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:24.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:23.829+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:24.012 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:23 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:23.829+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:25.065 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:24.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6806 osd.0 since back 2024-09-10T18:42:25.458030+0000 front 2024-09-10T18:42:25.458014+0000 (oldest deadline 2024-09-10T18:42:49.557942+0000) 2024-09-10T18:43:25.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:24.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6814 osd.1 since back 2024-09-10T18:42:32.359420+0000 front 2024-09-10T18:42:32.359398+0000 (oldest deadline 2024-09-10T18:42:56.959683+0000) 2024-09-10T18:43:25.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:24.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6822 osd.2 since back 2024-09-10T18:42:41.661040+0000 front 2024-09-10T18:42:41.660952+0000 (oldest deadline 2024-09-10T18:43:03.360899+0000) 2024-09-10T18:43:25.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:24.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.66:6830 osd.3 since back 2024-09-10T18:42:49.762042+0000 front 2024-09-10T18:42:49.762021+0000 (oldest deadline 2024-09-10T18:43:15.062013+0000) 2024-09-10T18:43:25.066 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:24 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7[44178]: 2024-09-10T18:43:24.805+0000 7f2726d04640 -1 osd.7 71 heartbeat_check: no reply from 172.21.15.117:6804 osd.4 since back 2024-09-10T18:42:55.062500+0000 front 2024-09-10T18:42:55.062532+0000 (oldest deadline 2024-09-10T18:43:19.762521+0000) 2024-09-10T18:43:26.061 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:25 smithi117 podman[57108]: 2024-09-10 18:43:25.782847428 +0000 UTC m=+5.439701193 container died cc66b0e0fbcc1388020d241e94aa1f1c5cf22d4ed7f700eae4a35745d3765a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T18:43:26.315 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:26 smithi117 podman[57108]: 2024-09-10 18:43:26.259345745 +0000 UTC m=+5.916199535 container cleanup cc66b0e0fbcc1388020d241e94aa1f1c5cf22d4ed7f700eae4a35745d3765a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T18:43:26.315 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:26 smithi117 bash[57108]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7 2024-09-10T18:43:26.815 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:26 smithi117 podman[57778]: 2024-09-10 18:43:26.493943055 +0000 UTC m=+0.706833911 container remove cc66b0e0fbcc1388020d241e94aa1f1c5cf22d4ed7f700eae4a35745d3765a31 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T18:43:27.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:26 smithi117 podman[57869]: 2024-09-10 18:43:26.723434577 +0000 UTC m=+0.015608897 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:43:27.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:26 smithi117 podman[57869]: 2024-09-10 18:43:26.83622286 +0000 UTC m=+0.128397168 container create dbeef19a007240f90e73a701d974d1bdc409e82b199ffafc7b20ce65c9dea466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T18:43:27.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:27 smithi117 podman[57869]: 2024-09-10 18:43:27.030544363 +0000 UTC m=+0.322718688 container init dbeef19a007240f90e73a701d974d1bdc409e82b199ffafc7b20ce65c9dea466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T18:43:27.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:27 smithi117 podman[57869]: 2024-09-10 18:43:27.034196196 +0000 UTC m=+0.326370509 container start dbeef19a007240f90e73a701d974d1bdc409e82b199ffafc7b20ce65c9dea466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T18:43:27.106 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:27 smithi117 podman[57869]: 2024-09-10 18:43:27.105120718 +0000 UTC m=+0.397295046 container attach dbeef19a007240f90e73a701d974d1bdc409e82b199ffafc7b20ce65c9dea466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T18:43:27.493 INFO:journalctl@ceph.osd.7.smithi117.stdout:Sep 10 18:43:27 smithi117 podman[57869]: 2024-09-10 18:43:27.210410465 +0000 UTC m=+0.502584783 container died dbeef19a007240f90e73a701d974d1bdc409e82b199ffafc7b20ce65c9dea466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-10T18:43:27.784 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T18:43:27.823 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:27.823 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T18:43:27.823 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T18:43:27.823 DEBUG:teuthology.orchestra.run.smithi066:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T18:43:28.228 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:27 smithi066 systemd[1]: Stopping Ceph rgw.foo.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:28.728 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:28 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a[50527]: 2024-09-10T18:43:28.230+0000 7f485cc94640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:43:28.728 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:28 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a[50527]: 2024-09-10T18:43:28.230+0000 7f486099b7c0 -1 shutting down 2024-09-10T18:43:38.581 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:38 smithi066 bash[65597]: time="2024-09-10T18:43:38Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:43:38.581 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:38 smithi066 podman[65597]: 2024-09-10 18:43:38.248223168 +0000 UTC m=+10.223095134 container died e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T18:43:38.964 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:38 smithi066 podman[65597]: 2024-09-10 18:43:38.737074872 +0000 UTC m=+10.711946882 container cleanup e5885dd8287e12b406ae6548b839f6291485ee130d14778ed2c76801360f9233 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-10T18:43:38.964 INFO:journalctl@ceph.rgw.foo.a.smithi066.stdout:Sep 10 18:43:38 smithi066 bash[65597]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T18:43:39.093 DEBUG:teuthology.orchestra.run.smithi066:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T18:43:39.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:39.131 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T18:43:39.131 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T18:43:39.131 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-f248e234-6fa2-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T18:43:39.417 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 systemd[1]: Stopping Ceph prometheus.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:43:39.816 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.436Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:43:39.817 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.437Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:43:39.817 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.437Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:43:39.817 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.437Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:43:39.817 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a[53438]: ts=2024-09-10T18:43:39.437Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:43:39.817 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:39 smithi117 podman[58087]: 2024-09-10 18:43:39.625916481 +0000 UTC m=+0.310448165 container died 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:40.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:40 smithi117 podman[58087]: 2024-09-10 18:43:40.06233727 +0000 UTC m=+0.746868953 container cleanup 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:40.272 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:40 smithi117 bash[58087]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:43:40.565 INFO:journalctl@ceph.prometheus.a.smithi117.stdout:Sep 10 18:43:40 smithi117 podman[58116]: 2024-09-10 18:43:40.271881578 +0000 UTC m=+0.825512501 container remove 42e1447df3a3fe625e9d92816cacfed94e7bfaaa395747488668940e36686179 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:40.848 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f248e234-6fa2-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T18:43:40.885 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T18:43:40.885 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T18:43:40.886 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f248e234-6fa2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:43:41.891 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:41 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:43:41.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi066.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.66:8443: connect: connection refused" 2024-09-10T18:43:41.891 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:41 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:43:41.628Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi117.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.117:8443: connect: connection refused" 2024-09-10T18:43:43.478 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:43 smithi066 systemd[1]: Stopping Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:43.917 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:43 smithi066 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a[60477]: ts=2024-09-10T18:43:43.600Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:43:43.917 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:43 smithi066 podman[66139]: 2024-09-10 18:43:43.621008212 +0000 UTC m=+0.224748450 container died 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:43.918 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:43 smithi066 podman[66139]: 2024-09-10 18:43:43.916627892 +0000 UTC m=+0.520368127 container cleanup 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:44.218 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:43 smithi066 bash[66139]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:43:44.219 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:44 smithi066 podman[66150]: 2024-09-10 18:43:44.134319706 +0000 UTC m=+0.512236443 container remove 4575e2a9bcd0709474d7a95656d52ef406f46c15f5c247e4ce0809b18c93685f (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:44.219 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:44 smithi066 podman[66150]: 2024-09-10 18:43:44.218094962 +0000 UTC m=+0.596011701 volume remove bd5e8d442b029cab540aa8fcc1d6c5432609e3950aaf3ce714bc0035ba31cd9a 2024-09-10T18:43:44.478 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:44 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:43:44.478 INFO:journalctl@ceph.alertmanager.a.smithi066.stdout:Sep 10 18:43:44 smithi066 systemd[1]: Stopped Ceph alertmanager.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:43:44.807 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:44 smithi066 systemd[1]: Stopping Ceph node-exporter.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:45.080 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:44 smithi066 podman[66258]: 2024-09-10 18:43:44.806722587 +0000 UTC m=+0.116073524 container died d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:45.439 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 podman[66258]: 2024-09-10 18:43:45.162867362 +0000 UTC m=+0.472218296 container cleanup d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:45.439 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 bash[66258]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T18:43:45.728 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 podman[66270]: 2024-09-10 18:43:45.438398234 +0000 UTC m=+0.635101270 container remove d0e424adf5c8aa3a41f8127284049964162e8ba562c2768dcb8310277d1c2823 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:43:45.728 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:43:45.728 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T18:43:45.728 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 systemd[1]: Stopped Ceph node-exporter.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:43:45.728 INFO:journalctl@ceph.node-exporter.a.smithi066.stdout:Sep 10 18:43:45 smithi066 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 2.006s CPU time. 2024-09-10T18:43:46.367 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f248e234-6fa2-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T18:43:48.065 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:47 smithi117 systemd[1]: Stopping Ceph iscsi.iscsi.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:43:48.565 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:48 smithi117 ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a[47027]: debug Shutdown received 2024-09-10T18:43:58.503 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 bash[59144]: time="2024-09-10T18:43:58Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T18:43:58.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 podman[59144]: 2024-09-10 18:43:58.233446685 +0000 UTC m=+10.121617089 container died 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True) 2024-09-10T18:43:58.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 podman[59144]: 2024-09-10 18:43:58.668574658 +0000 UTC m=+10.556745110 container cleanup 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True) 2024-09-10T18:43:58.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 bash[59144]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T18:43:59.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 podman[59158]: 2024-09-10 18:43:58.878437569 +0000 UTC m=+0.642290932 container remove 04044b78827c11e0e58ab86fe856b8d7a173980602717f76d2f7d4321bd3e6b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:43:59.176 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:58 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T18:43:59.441 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 podman[59202]: 2024-09-10 18:43:59.174941884 +0000 UTC m=+0.092814339 container died 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:43:59.815 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 podman[59202]: 2024-09-10 18:43:59.622612785 +0000 UTC m=+0.540485240 container cleanup 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 bash[59202]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 46797 (conmon) remains running after unit stopped. 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 59214 (podman) remains running after unit stopped. 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 systemd[1]: Stopped Ceph iscsi.iscsi.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:43:59.816 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 2.441s CPU time. 2024-09-10T18:44:00.313 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 systemd[1]: Stopping Ceph grafana.a for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:44:00.314 INFO:journalctl@ceph.iscsi.iscsi.a.smithi117.stdout:Sep 10 18:43:59 smithi117 podman[59214]: 2024-09-10 18:43:59.932848056 +0000 UTC m=+0.761669977 container remove 644fe175bbc457fe1f036b1191604a5f8d73b0dbb998e92990e161f8486c6c77 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T18:44:00.565 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 podman[59337]: 2024-09-10 18:44:00.312546949 +0000 UTC m=+0.110336996 container died d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, vendor=Red Hat, Inc., maintainer=Paul Cuzner , com.redhat.component=ubi8-container, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.expose-services=, release=239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.buildah.version=1.32.0, architecture=x86_64, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-type=git, version=8.5, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public) 2024-09-10T18:44:00.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 podman[59337]: 2024-09-10 18:44:00.685127374 +0000 UTC m=+0.482917420 container cleanup d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, io.openshift.tags=base rhel8, description=Ceph Grafana Container, maintainer=Paul Cuzner , io.openshift.expose-services=, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-type=git, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., architecture=x86_64, build-date=2022-04-29T11:48:11.055897, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., distribution-scope=public) 2024-09-10T18:44:00.921 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 bash[59337]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a 2024-09-10T18:44:01.244 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 podman[59350]: 2024-09-10 18:44:00.919704524 +0000 UTC m=+0.605075329 container remove d893f555d22af6b005f45036ec08d0c27c3122af15ea9e0a8a179991d279fd68 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.openshift.tags=base rhel8, vendor=Red Hat, Inc., vcs-type=git, com.redhat.component=ubi8-container, release=239.1651231664, io.buildah.version=1.32.0, distribution-scope=public, build-date=2022-04-29T11:48:11.055897, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, io.openshift.expose-services=, maintainer=Paul Cuzner , architecture=x86_64, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-10T18:44:01.244 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:00 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:44:01.244 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:01 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T18:44:01.244 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:01 smithi117 systemd[1]: Stopped Ceph grafana.a for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:44:01.244 INFO:journalctl@ceph.grafana.a.smithi117.stdout:Sep 10 18:44:01 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@grafana.a.service: Consumed 4.772s CPU time. 2024-09-10T18:44:01.566 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:01 smithi117 systemd[1]: Stopping Ceph node-exporter.b for f248e234-6fa2-11ef-bcea-c7b262605968... 2024-09-10T18:44:01.566 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:01 smithi117 podman[59458]: 2024-09-10 18:44:01.491924504 +0000 UTC m=+0.114200001 container died 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:44:02.065 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:01 smithi117 podman[59458]: 2024-09-10 18:44:01.830745611 +0000 UTC m=+0.453021105 container cleanup 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:44:02.066 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:01 smithi117 bash[59458]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T18:44:02.408 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:02 smithi117 podman[59471]: 2024-09-10 18:44:02.115195794 +0000 UTC m=+0.627502957 container remove 9f4428ee0322af06b7ef0f767689cd4cf85127dd3821716f369663356492781b (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-f248e234-6fa2-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:44:02.409 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:02 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T18:44:02.409 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:02 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T18:44:02.409 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:02 smithi117 systemd[1]: Stopped Ceph node-exporter.b for f248e234-6fa2-11ef-bcea-c7b262605968. 2024-09-10T18:44:02.409 INFO:journalctl@ceph.node-exporter.b.smithi117.stdout:Sep 10 18:44:02 smithi117 systemd[1]: ceph-f248e234-6fa2-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 1.844s CPU time. 2024-09-10T18:44:02.909 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:44:02.944 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:44:02.971 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T18:44:02.973 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi066/crash 2024-09-10T18:44:02.974 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:44:03.023 INFO:teuthology.orchestra.run.smithi066.stderr:tar: /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:44:03.023 INFO:teuthology.orchestra.run.smithi066.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:44:03.025 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi117/crash 2024-09-10T18:44:03.026 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash -- . 2024-09-10T18:44:03.069 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T18:44:03.069 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-09-10T18:44:03.070 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T18:44:03.071 DEBUG:teuthology.orchestra.run.smithi066:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v MON_DOWN | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | head -n 1 2024-09-10T18:44:03.100 INFO:teuthology.orchestra.run.smithi066.stderr:grep: /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T18:44:03.102 INFO:tasks.cephadm:Compressing logs... 2024-09-10T18:44:03.102 DEBUG:teuthology.orchestra.run.smithi066:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:44:03.144 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:: No such file or directory 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/tcmu-runner.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr: 62.9% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T18:44:03.170 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T18:44:03.171 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T18:44:03.171 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T18:44:03.171 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T18:44:03.174 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:44:03.174 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:: No such file or directory 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.c.log: /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T18:44:03.176 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T18:44:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 58.5% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T18:44:03.177 INFO:teuthology.orchestra.run.smithi066.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T18:44:03.177 INFO:teuthology.orchestra.run.smithi117.stderr: 90.7% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T18:44:03.191 INFO:teuthology.orchestra.run.smithi066.stderr:/var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T18:44:03.219 INFO:teuthology.orchestra.run.smithi117.stderr: 90.2% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:44:03.226 INFO:teuthology.orchestra.run.smithi066.stderr: 90.2% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T18:44:03.299 INFO:teuthology.orchestra.run.smithi117.stderr: 92.2% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T18:44:03.308 INFO:teuthology.orchestra.run.smithi066.stderr: 87.6% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T18:44:03.321 INFO:teuthology.orchestra.run.smithi066.stderr: 92.4% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T18:44:03.325 INFO:teuthology.orchestra.run.smithi117.stderr: 93.2% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T18:44:03.361 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T18:44:03.385 INFO:teuthology.orchestra.run.smithi066.stderr: 93.1% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T18:44:03.387 INFO:teuthology.orchestra.run.smithi117.stderr: 93.1% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T18:44:03.392 INFO:teuthology.orchestra.run.smithi117.stderr: 93.4% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T18:44:03.394 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-09-10T18:44:03.394 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m0.235s 2024-09-10T18:44:03.395 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m0.946s 2024-09-10T18:44:03.395 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m0.054s 2024-09-10T18:44:03.401 INFO:teuthology.orchestra.run.smithi066.stderr: 93.3% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T18:44:03.437 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T18:44:03.484 INFO:teuthology.orchestra.run.smithi066.stderr: 93.4% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T18:44:03.563 INFO:teuthology.orchestra.run.smithi066.stderr: 91.3% -- replaced with /var/log/ceph/f248e234-6fa2-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T18:44:03.565 INFO:teuthology.orchestra.run.smithi066.stderr: 2024-09-10T18:44:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:real 0m0.407s 2024-09-10T18:44:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:user 0m1.709s 2024-09-10T18:44:03.565 INFO:teuthology.orchestra.run.smithi066.stderr:sys 0m0.077s 2024-09-10T18:44:03.566 INFO:tasks.cephadm:Archiving logs... 2024-09-10T18:44:03.566 DEBUG:teuthology.misc:Transferring archived files from smithi066:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi066/log 2024-09-10T18:44:03.567 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:44:03.848 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi117/log 2024-09-10T18:44:03.849 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T18:44:04.008 INFO:tasks.cephadm:Removing cluster... 2024-09-10T18:44:04.009 DEBUG:teuthology.orchestra.run.smithi066:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f248e234-6fa2-11ef-bcea-c7b262605968 --force 2024-09-10T18:44:04.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f248e234-6fa2-11ef-bcea-c7b262605968 --force 2024-09-10T18:44:04.713 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T18:44:04.714 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:44:04.730 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T18:44:04.746 INFO:tasks.cephadm:Teardown complete 2024-09-10T18:44:04.746 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-09-10T18:44:04.760 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi066:/dev/vg_nvme/lv_1... 2024-09-10T18:44:04.760 DEBUG:teuthology.orchestra.run.smithi066:> sudo nvme disconnect -n lv_1 2024-09-10T18:44:04.927 INFO:teuthology.orchestra.run.smithi066.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-10T18:44:04.928 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi066:/dev/vg_nvme/lv_2... 2024-09-10T18:44:04.929 DEBUG:teuthology.orchestra.run.smithi066:> sudo nvme disconnect -n lv_2 2024-09-10T18:44:05.080 INFO:teuthology.orchestra.run.smithi066.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-10T18:44:05.081 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi066:/dev/vg_nvme/lv_3... 2024-09-10T18:44:05.081 DEBUG:teuthology.orchestra.run.smithi066:> sudo nvme disconnect -n lv_3 2024-09-10T18:44:05.230 INFO:teuthology.orchestra.run.smithi066.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-10T18:44:05.231 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi066:/dev/vg_nvme/lv_4... 2024-09-10T18:44:05.231 DEBUG:teuthology.orchestra.run.smithi066:> sudo nvme disconnect -n lv_4 2024-09-10T18:44:05.378 INFO:teuthology.orchestra.run.smithi066.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-10T18:44:05.379 DEBUG:teuthology.orchestra.run.smithi066:> set -ex 2024-09-10T18:44:05.379 DEBUG:teuthology.orchestra.run.smithi066:> sudo dd of=/scratch_devs 2024-09-10T18:44:05.405 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi117:/dev/vg_nvme/lv_1... 2024-09-10T18:44:05.405 DEBUG:teuthology.orchestra.run.smithi117:> sudo nvme disconnect -n lv_1 2024-09-10T18:44:05.559 INFO:teuthology.orchestra.run.smithi117.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-09-10T18:44:05.561 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi117:/dev/vg_nvme/lv_2... 2024-09-10T18:44:05.561 DEBUG:teuthology.orchestra.run.smithi117:> sudo nvme disconnect -n lv_2 2024-09-10T18:44:05.705 INFO:teuthology.orchestra.run.smithi117.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-09-10T18:44:05.707 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi117:/dev/vg_nvme/lv_3... 2024-09-10T18:44:05.707 DEBUG:teuthology.orchestra.run.smithi117:> sudo nvme disconnect -n lv_3 2024-09-10T18:44:05.874 INFO:teuthology.orchestra.run.smithi117.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-09-10T18:44:05.876 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi117:/dev/vg_nvme/lv_4... 2024-09-10T18:44:05.876 DEBUG:teuthology.orchestra.run.smithi117:> sudo nvme disconnect -n lv_4 2024-09-10T18:44:06.030 INFO:teuthology.orchestra.run.smithi117.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-09-10T18:44:06.032 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-09-10T18:44:06.032 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/scratch_devs 2024-09-10T18:44:06.056 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T18:44:06.065 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T18:44:06.065 DEBUG:teuthology.orchestra.run.smithi066:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:44:06.068 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:44:06.082 INFO:teuthology.orchestra.run.smithi066.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:44:06.086 INFO:teuthology.orchestra.run.smithi066.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:44:06.086 INFO:teuthology.orchestra.run.smithi066.stdout:=============================================================================== 2024-09-10T18:44:06.087 INFO:teuthology.orchestra.run.smithi066.stdout:^* hv01.front.sepia.ceph.com 3 6 377 27 +90us[ +83us] +/- 51ms 2024-09-10T18:44:06.087 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 28 +898us[ +890us] +/- 65ms 2024-09-10T18:44:06.087 INFO:teuthology.orchestra.run.smithi066.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -1051us[-1059us] +/- 64ms 2024-09-10T18:44:06.087 INFO:teuthology.orchestra.run.smithi066.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:44:06.110 INFO:teuthology.orchestra.run.smithi117.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:=============================================================================== 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +76us[ +64us] +/- 51ms 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 26 +871us[ +871us] +/- 65ms 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -1060us[-1060us] +/- 64ms 2024-09-10T18:44:06.115 INFO:teuthology.orchestra.run.smithi117.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:44:06.115 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T18:44:06.124 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T18:44:06.125 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T18:44:06.133 DEBUG:teuthology.orchestra.run.smithi066:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:44:06.212 DEBUG:teuthology.orchestra.run.smithi117:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T18:44:06.279 DEBUG:teuthology.orchestra.run.smithi066:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:44:06.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:44:06.311 DEBUG:teuthology.orchestra.run.smithi117:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:44:06.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:44:06.339 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T18:44:06.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T18:44:06.356 INFO:teuthology.task.internal:Duration was 972.081141 seconds 2024-09-10T18:44:06.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T18:44:06.366 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T18:44:06.367 DEBUG:teuthology.orchestra.run.smithi066:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:44:06.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T18:44:06.410 INFO:teuthology.orchestra.run.smithi066.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:44:06.418 INFO:teuthology.orchestra.run.smithi117.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:44:06.849 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T18:44:06.849 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi066.front.sepia.ceph.com 2024-09-10T18:44:06.849 DEBUG:teuthology.orchestra.run.smithi066:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:44:06.873 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-09-10T18:44:06.873 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T18:44:06.913 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T18:44:06.914 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:44:06.916 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T18:44:06.979 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T18:44:06.980 DEBUG:teuthology.orchestra.run.smithi066:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:44:06.986 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T18:44:07.267 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T18:44:07.277 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T18:44:07.277 DEBUG:teuthology.orchestra.run.smithi066:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:44:07.303 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T18:44:07.328 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T18:44:07.338 DEBUG:teuthology.orchestra.run.smithi066:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:44:07.345 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:44:07.367 INFO:teuthology.orchestra.run.smithi066.stdout:kernel.core_pattern = core 2024-09-10T18:44:07.392 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-09-10T18:44:07.436 DEBUG:teuthology.orchestra.run.smithi066:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:44:07.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:44:07.460 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T18:44:07.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:44:07.491 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T18:44:07.504 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T18:44:07.506 DEBUG:teuthology.misc:Transferring archived files from smithi066:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi066 2024-09-10T18:44:07.506 DEBUG:teuthology.orchestra.run.smithi066:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:44:07.547 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900234/remote/smithi117 2024-09-10T18:44:07.547 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T18:44:07.583 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T18:44:07.584 DEBUG:teuthology.orchestra.run.smithi066:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:44:07.586 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T18:44:07.633 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T18:44:07.645 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T18:44:07.645 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T18:44:07.654 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T18:44:07.655 DEBUG:teuthology.orchestra.run.smithi066:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:44:07.657 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T18:44:07.671 INFO:teuthology.orchestra.run.smithi066.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:44 /home/ubuntu/cephtest 2024-09-10T18:44:07.688 INFO:teuthology.orchestra.run.smithi117.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 18:44 /home/ubuntu/cephtest 2024-09-10T18:44:07.690 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T18:44:07.700 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T18:44:07.722 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke/{0-distro/centos_9.stream 0-nvme-loop agent/on fixed-2 mon_election/classic start} duration: 972.0811405181885 owner: scheduled_gabrioux@teuthology success: true 2024-09-10T18:44:07.722 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:44:07.794 INFO:teuthology.run:pass